Skip to content

Refactor pyreverse Association Logic #10397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 39 commits into
base: main
Choose a base branch
from

Conversation

Julfried
Copy link
Contributor

@Julfried Julfried commented May 18, 2025

Type of Changes

Type
βœ“ πŸ› Bug fix
βœ“ πŸ”¨ Refactoring

Description

Closes #9045

Summary

Fixed incorrect UML semantics in pyreverse for aggregation vs composition relationships. The previous implementation had no explicit CompositionsHandler class and did not differentiate between Aggregation and Composition properly, treating most relationships as aggregation by default.

Problem

Pyreverse previously lacked proper UML relationship semantics. So I would proposes the following distinction for Python class relationships:

fields.py

# Test for https://github.com/pylint-dev/pylint/issues/9045

class P:
    pass

class A:
    x: P  # just type hint, no ownership β†’ Association

class B:
    def __init__(self, x: P):
        self.x = x  # receives object, not created β†’ Aggregation

class C:
    x: P
    def __init__(self, x: P):
        self.x = x  # receives object, not created β†’ Aggregation

class D:
    x: P
    def __init__(self):
        self.x = P()  # creates object β†’ Composition

class E:
    def __init__(self):
        self.x = P()  # creates object β†’ Composition

fields.mmd

classDiagram
  class A {
    x
  }
  class B {
    x
  }
  class C {
    x
  }
  class D {
    x
  }
  class E {
    x
  }
  class P {
  }
  P --> A : x
  P --* D : x
  P --* E : x
  P --o B : x
  P --o C : x

Changes Made

1. Added CompositionsHandler

Created dedicated handler following chain of responsibility pattern:
CompositionsHandler β†’ AggregationsHandler β†’ AssociationsHandler

2. Updated Printers

  • Added EdgeType.COMPOSITION enum value
  • Updated MermaidJS and PlantUML printers with correct arrows per Mermaid docs
  • For the Dot printer I used plain arrows, but I did not manage to find something in the dot language documentation

3. Prevented Duplicate Relationships

Modified extract_relationships() to process relationships by priority and avoid duplicates.

Currently I only made the modifications needed to implement the new Associations Logic. If you like the proposed distinction I would work on making the tests pass :)

@Julfried Julfried requested a review from DudeNr33 as a code owner May 18, 2025 23:51

This comment has been minimized.

@Pierre-Sassoulas Pierre-Sassoulas added pyreverse Related to pyreverse component Enhancement ✨ Improvement to a component Work in progress labels May 19, 2025
Copy link
Collaborator

@DudeNr33 DudeNr33 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the addition! πŸ‘

I only left two remarks for cases where at least from the comment it seems like the relationship is processed by the wrong handler. Am I missing something?

It would be great if you can add tests for all possible output formats as all printers where changed. You can specify multiple output formats for a single functional test in the [testoptions].

@Julfried
Copy link
Contributor Author

Thanks for the review. I agree that the logic is not complete and it needs some more work and I need to figure out all the possible edge cases. Also I think some of the other tests have changed , and I need to rebase aswell. I think this will take some time, but lets see how it goes :)

This comment has been minimized.

@DudeNr33
Copy link
Collaborator

DudeNr33 commented Jun 2, 2025

@Julfried please ping me once the PR is ready to be reviewed again. No hurry though, just want to point this out as I am not actively monitoring the open PRs and instead rely more on the email notifications. πŸ™‚

@jacobtylerwalls jacobtylerwalls changed the title Refactor pyerverse Association Logic Refactor pyreverse Association Logic Jun 7, 2025
Julfried added 11 commits June 14, 2025 10:01
…o that the differentation between aggregation and composition is correct according to UMLEnhance composition and aggregation handling in AST node processing so that the differentation between aggregation and composition is correct according to UML
…ons for extracting element types and resolving class definitionsEnhance type resolution in AssociationsHandler and add utility functions for extracting element types and resolving class definitions

This comment has been minimized.

Copy link

codecov bot commented Jun 29, 2025

Codecov Report

Attention: Patch coverage is 98.26087% with 2 lines in your changes missing coverage. Please review.

Project coverage is 95.88%. Comparing base (25a0f9e) to head (d275c2f).
Report is 34 commits behind head on main.

Files with missing lines Patch % Lines
pylint/pyreverse/inspector.py 97.97% 2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (98.26%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #10397      +/-   ##
==========================================
- Coverage   95.91%   95.88%   -0.03%     
==========================================
  Files         176      176              
  Lines       19147    19227      +80     
==========================================
+ Hits        18364    18436      +72     
- Misses        783      791       +8     
Files with missing lines Coverage Ξ”
pylint/pyreverse/diagrams.py 92.85% <100.00%> (-0.76%) ⬇️
pylint/pyreverse/dot_printer.py 85.71% <ΓΈ> (ΓΈ)
pylint/pyreverse/mermaidjs_printer.py 98.43% <ΓΈ> (ΓΈ)
pylint/pyreverse/plantuml_printer.py 100.00% <ΓΈ> (ΓΈ)
pylint/pyreverse/printer.py 100.00% <100.00%> (ΓΈ)
pylint/pyreverse/writer.py 98.93% <100.00%> (-1.07%) ⬇️
pylint/pyreverse/inspector.py 86.11% <97.97%> (+4.89%) ⬆️

... and 11 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This comment has been minimized.

This comment has been minimized.

@Julfried
Copy link
Contributor Author

@DudeNr33 Finally ready for review again. All functional tests are passing and the association logic is working correctly now.

Interestingly, I figured while playing around that this PR also fixes #10333 (duplicate arrows issue) as a side effect. The root cause was the same - improper relationship classification led to the same attribute being processed multiple times across different relationship types. Our priority-based processing with duplicate prevention solves this at the source.

This example from #10333:

class A:
    def __init__(self) -> None:
        self.var = 2

class B:
    def __init__(self) -> None:
        self.a_obj = A()

    def func(self):
        self.a_obj = A()
        self.a_obj = A()

Now correctly produces single composition relationship instead of duplicates:

classDiagram
  class A {
    var : int
  }
  class B {
    a_obj
    func()
  }
  A --* B : a_obj

Should I add the functional test cases from #10333 to this PR, or handle them in a follow-up? The PR is getting quite large, but its up to you.

The core association logic is solid and ready for review either way.

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, that looks pretty good !

@@ -0,0 +1,3 @@
Enhanced pyreverse to properly distinguish between UML relationship types (association, aggregation, composition) based on object ownership semantics. Type annotations without assignment are now treated as associations, parameter assignments as aggregations, and object instantiation as compositions.

Closes #9045
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you acccidentally fixed multiple issues, you can close more then one :D

@Pierre-Sassoulas
Copy link
Member

I think it's better to add functional test cases for the other issue here as you fixed it here. Usually we do another MR because we don't realise an issue is fixed until later.

Copy link
Contributor

πŸ€– Effect of this PR on checked open source code: πŸ€–

Effect on home-assistant:
The following messages are now emitted:

  1. import-private-name:
    Imported private module (_collections_abc)
    https://github.com/home-assistant/core/blob/e210681751249c1b92fb74adfdaa72baf9a4ccc8/homeassistant/components/smlight/binary_sensor.py#L5

Effect on music21:
The following messages are now emitted:

  1. invalid-name:
    Attribute name "id" doesn't conform to '[a-z_][A-Za-z0-9_]{2,30}$' pattern
    https://github.com/cuthbertLab/music21/blob/f2e09f14916ece7651122d160a43b8950608182b/music21/base.py#L612
  2. redefined-variable-type:
    Redefinition of sc3 type from music21.scale.MinorScale to music21.scale.MajorScale
    https://github.com/cuthbertLab/music21/blob/f2e09f14916ece7651122d160a43b8950608182b/music21/scale/test_scale_main.py#L163

The following messages are no longer emitted:

  1. invalid-name:
    Attribute name "id" doesn't conform to '[a-z_][A-Za-z0-9_]{2,30}$' pattern
    https://github.com/cuthbertLab/music21/blob/f2e09f14916ece7651122d160a43b8950608182b/music21/prebase.py#L293
  2. redefined-variable-type:
    Redefinition of sc3 type from music21.scale.MajorScale to music21.scale.MinorScale
    https://github.com/cuthbertLab/music21/blob/f2e09f14916ece7651122d160a43b8950608182b/music21/scale/test_scale_main.py#L169

Effect on psycopg:
The following messages are now emitted:

  1. c-extension-no-member:
    Module 'psycopg.pq._pq_ctypes' has no 'PQnoticeReceiver' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects.
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L55
  2. too-many-function-args:
    Too many positional arguments for function call
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L360

The following messages are no longer emitted:

  1. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L545
  2. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L562
  3. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L622
  4. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L712
  5. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L730
  6. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L773
  7. unreachable:
    Unreachable code
    https://github.com/psycopg/psycopg/blob/ec15df110bcc25a63099cd22731f3b4bdd01cbb0/psycopg/psycopg/pq/pq_ctypes.py#L990

This comment was generated for commit d275c2f

@Julfried
Copy link
Contributor Author

Julfried commented Jun 30, 2025

I updated the functional tests, so now this also Closes #9267

The example from #8046 currently produces this output:

@startuml classes_toy_code
set namespaceSeparator none
class "ExampleClass" as parser.toy_code.ExampleClass {
  example1 : Optional[int]
  example2 : Optional[int]
}
@enduml

However I think this is not the output the was discussed in the issue, since the class level attributes are still not handled correctly.

Let me know what you think :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement ✨ Improvement to a component pyreverse Related to pyreverse component Work in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pyreverse: composition / aggregation arrow strange behavior (and field annotation bug)
3 participants