Skip to content

Conversation

cxzhong
Copy link
Contributor

@cxzhong cxzhong commented Oct 13, 2025

tomli is a TOML parser that was needed for Python < 3.11. Since Python 3.11+, tomllib is part of the standard library. tomli-w (TOML writer) is kept as it's still needed for writing TOML files.

Changes:

  • Removed build/pkgs/tomli directory
  • Updated package dependencies to remove tomli references
  • Remove the unused 3.10 env file

📝 Checklist

  • The title is concise and informative.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation and checked the documentation preview.

⌛ Dependencies

tomli is a TOML parser that was needed for Python < 3.11.
Since Python 3.11+, tomllib is part of the standard library.
tomli-w (TOML writer) is kept as it's still needed for writing TOML files.

Changes:
- Removed tomli from all environment-*.yml files
- Removed build/pkgs/tomli directory
- Updated package dependencies to remove tomli references
@cxzhong cxzhong requested a review from tobiasdiez October 13, 2025 07:52
Copy link

github-actions bot commented Oct 13, 2025

Documentation preview for this PR (built with commit b123481; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@cxzhong cxzhong marked this pull request as ready for review October 13, 2025 09:07
@Copilot Copilot AI review requested due to automatic review settings October 13, 2025 09:07
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR removes the tomli dependency from the codebase as it's no longer needed for Python 3.11+. Since Python 3.11 includes tomllib in the standard library for parsing TOML files, the external tomli package is redundant. The tomli-w package is retained as it's still needed for writing TOML files.

  • Removed tomli from all environment configuration files across different Python versions and platforms
  • Deleted the entire build/pkgs/tomli directory and its associated files
  • Updated package dependency files to remove tomli references

Reviewed Changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated no comments.

File Description
environment-*.yml Removed tomli dependency from conda environment files for Python 3.10-3.13 across all platforms
build/pkgs/tomli/ Completely removed tomli package directory and all configuration files
build/pkgs/*/dependencies Updated dependency lists to remove tomli references from various packages
Comments suppressed due to low confidence (1)

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

@tobiasdiez tobiasdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conda env files are automatically generated. So if tomli is in there, it means that another python library still needs it (or the metadata in some conda package needs to be updated to reflect that tomli is no longer needed).

Cannot say much about the changes in build. If all of those pkgs are no longer actually using tomli, then it would be fine to remove it.

@cxzhong
Copy link
Contributor Author

cxzhong commented Oct 13, 2025

The conda env files are automatically generated. So if tomli is in there, it means that another python library still needs it (or the metadata in some conda package needs to be updated to reflect that tomli is no longer needed).

Cannot say much about the changes in build. If all of those pkgs are no longer actually using tomli, then it would be fine to remove it.

Maybe some packages in that time also support the python <3.11. So it needs the tomli package. Maybe we need to generate new conda env files. Thank you very much.
(edit: meson-python in conda source relies on tomli even if python>3.11, I finally found)

@cxzhong cxzhong marked this pull request as draft October 13, 2025 10:37
@cxzhong
Copy link
Contributor Author

cxzhong commented Oct 13, 2025

 FAILED: [code=1] src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/meson-generated_src_sage_libs_linbox_linbox_flint_interface.pyx.cpp.o
  ccache arm64-apple-darwin20.0.0-clang++ -Isrc/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p -Isrc/sage/libs/linbox -I../src/sage/libs/linbox -Isrc/sage/libs/flint -I../src/sage/libs/flint -I/Users/runner/miniconda3/envs/sage-dev/include/python3.12 -I/Users/runner/miniconda3/envs/sage-dev/include -fvisibility=hidden -fvisibility-inlines-hidden -fdiagnostics-color=always -DNDEBUG -Wall -Winvalid-pch -std=c++11 -O3 -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem /Users/runner/miniconda3/envs/sage-dev/include -D_FORTIFY_SOURCE=2 -isystem /Users/runner/miniconda3/envs/sage-dev/include -DDISABLE_COMMENTATOR -pthread -DFFLAS_COMPILED -DFFPACK_COMPILED -F/Applications/Xcode_16.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Accelerate.framework -idirafter/Applications/Xcode_16.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Accelerate.framework/Headers -MD -MQ src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/meson-generated_src_sage_libs_linbox_linbox_flint_interface.pyx.cpp.o -MF src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/meson-generated_src_sage_libs_linbox_linbox_flint_interface.pyx.cpp.o.d -o src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/meson-generated_src_sage_libs_linbox_linbox_flint_interface.pyx.cpp.o -c src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/src/sage/libs/linbox/linbox_flint_interface.pyx.cpp
  In file included from src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/src/sage/libs/linbox/linbox_flint_interface.pyx.cpp:1154:
  In file included from /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/dense-matrix.h:85:
  In file included from /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/densematrix/blas-matrix.h:42:
  In file included from /Users/runner/miniconda3/envs/sage-dev/include/linbox/vector/blas-vector.h:47:
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/vector/blas-subvector.h:121:20: error: no member named 'data' in 'BlasSubvector<_Vector>'
    121 |             _ptr(V.data()+beg), _size(dim), _inc(inc), _field(&V.field()) {}
        |                  ~ ^
  In file included from src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/src/sage/libs/linbox/linbox_flint_interface.pyx.cpp:1156:
  In file included from /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparse-matrix.h:76:
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ell-matrix.h:1216:16: error: cannot assign to non-static data member '_ld' with const-qualified type 'const size_t &' (aka 'const unsigned long &')
   1216 |                                 _ld        = iter._ld ;
        |                                 ~~~        ^
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ell-matrix.h:1177:19: note: non-static data member '_ld' declared const here
   1177 |                         const size_t & _ld ;
        |                         ~~~~~~~~~~~~~~~^~~
  In file included from src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/src/sage/libs/linbox/linbox_flint_interface.pyx.cpp:1156:
  In file included from /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparse-matrix.h:77:
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ellr-matrix.h:1108:12: error: no viable overloaded '='
   1108 |                                 _rowid = iter._rowid;
        |                                 ~~~~~~ ^ ~~~~~~~~~~~
  /Users/runner/miniconda3/envs/sage-dev/bin/../include/c++/v1/vector:546:63: note: candidate function not viable: 'this' argument has type 'const std::vector<size_t>' (aka 'const vector<unsigned long>'), but method is not marked const
    546 |   _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI vector& operator=(const vector& __x);
        |                                                               ^
  /Users/runner/miniconda3/envs/sage-dev/bin/../include/c++/v1/vector:554:63: note: candidate function not viable: 'this' argument has type 'const std::vector<size_t>' (aka 'const vector<unsigned long>'), but method is not marked const
    554 |   _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI vector& operator=(initializer_list<value_type> __il) {
        |                                                               ^
  /Users/runner/miniconda3/envs/sage-dev/bin/../include/c++/v1/vector:569:63: note: candidate function not viable: 'this' argument has type 'const std::vector<size_t>' (aka 'const vector<unsigned long>'), but method is not marked const
    569 |   _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI vector& operator=(vector&& __x)
        |                                                               ^
  In file included from src/sage/libs/linbox/linbox_flint_interface.cpython-312-darwin.so.p/src/sage/libs/linbox/linbox_flint_interface.pyx.cpp:1156:
  In file included from /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparse-matrix.h:77:
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ellr-matrix.h:1109:9: error: cannot assign to non-static data member '_ld' with const-qualified type 'const size_t &' (aka 'const unsigned long &')
   1109 |                                 _ld = iter._ld ;
        |                                 ~~~ ^
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ellr-matrix.h:1075:19: note: non-static data member '_ld' declared const here
   1075 |                         const size_t & _ld ;
        |                         ~~~~~~~~~~~~~~~^~~
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ellr-matrix.h:1258:16: error: cannot assign to non-static data member '_ld' with const-qualified type 'const size_t &' (aka 'const unsigned long &')
   1258 |                                 _ld        = iter._ld ;
        |                                 ~~~        ^
  /Users/runner/miniconda3/envs/sage-dev/include/linbox/matrix/sparsematrix/sparse-ellr-matrix.h:1215:19: note: non-static data member '_ld' declared const here
   1215 |                         const size_t & _ld ;
        |                         ~~~~~~~~~~~~~~~^~~
  5 errors generated.

some errors in macos on python3.12 3.11. The version of clang is too high.

@cxzhong cxzhong closed this Oct 13, 2025
@cxzhong cxzhong deleted the remove-tomli branch October 13, 2025 11:10
@cxzhong cxzhong restored the remove-tomli branch October 13, 2025 11:19
@cxzhong cxzhong reopened this Oct 13, 2025
@cxzhong
Copy link
Contributor Author

cxzhong commented Oct 13, 2025

This is because the maxima version raise such errors, maxima version 5.48.1 use different expression.

@cxzhong cxzhong changed the title Remove tomli dependency TEST Oct 13, 2025
- Updated input hashes in environment files for consistency.
- Changed version of `brial` from 1.2.3 to 1.2.12 in Linux and macOS environments.
- Updated `libbrial` version from 1.2.3 to 1.2.12 in Linux and macOS environments.
- Adjusted `linbox` version from 1.7.1 to 1.7.0 in Linux and macOS environments.
- Changed `m4ri` version from 20250128 to 20140914 in Linux and updated `m4rie` version accordingly.
- Updated various dependencies in the Windows environment, removing obsolete packages and ensuring compatibility.
@tobiasdiez
Copy link
Contributor

You can base it on #39189, which fixes these compilation errors by excluding certain package versions.

@cxzhong
Copy link
Contributor Author

cxzhong commented Oct 13, 2025

You can base it on #39189, which fixes these compilation errors by excluding certain package versions.

Thank you very much. I will wait this merged. Now everything works.

- Updated `environment-3.13-win.yml` with new dependencies and version updates.
- Added several new packages including `annotated-types`, `backports`, `click`, and others.
- Updated Python version from 3.13.7 to 3.13.8 and adjusted related dependencies.
- Enhanced `update-conda.py` to use threading for processing platforms and Python versions concurrently.
- Refactored dependency retrieval logic to improve clarity and maintainability.
- Removed unnecessary comments and cleaned up the code for better readability.
@cxzhong cxzhong changed the title TEST Remove tomli Oct 15, 2025
@cxzhong
Copy link
Contributor Author

cxzhong commented Oct 15, 2025

I will wait for the #39189 merge

@cxzhong cxzhong marked this pull request as ready for review October 17, 2025 06:36
Removed 'toml' from the list of dependencies.
Copy link
Contributor

@tobiasdiez tobiasdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I cannot say much about the changes in the build folder.

@tobiasdiez tobiasdiez requested a review from dimpase October 18, 2025 01:15
@dimpase
Copy link
Member

dimpase commented Oct 18, 2025

looks OK, if CI agrees

@dimpase
Copy link
Member

dimpase commented Oct 18, 2025

there still tomli and tomli-w (not sure what the latter is) in yml files: (look at git grep tomli)

...
environment-3.12-macos-x86_64.yml:  - tomli=2.2.1=pyhe01879c_2
environment-3.12-macos-x86_64.yml:  - tomli-w=1.2.0=pyhd8ed1ab_0
...

and tomli-w in uv.lock

@cxzhong
Copy link
Contributor Author

cxzhong commented Oct 18, 2025

there still tomli and tomli-w (not sure what the latter is) in yml files: (look at git grep tomli)

...
environment-3.12-macos-x86_64.yml:  - tomli=2.2.1=pyhe01879c_2
environment-3.12-macos-x86_64.yml:  - tomli-w=1.2.0=pyhd8ed1ab_0
...

and tomli-w in uv.lock

because in conda, meson-python rely tomli for all Python versions. But in pyproject.toml, It only rely tomli for Python<3.11

vbraun pushed a commit to vbraun/sage that referenced this pull request Oct 19, 2025
sagemathgh-41036: Remove tomli
    
tomli is a TOML parser that was needed for Python < 3.11. Since Python
3.11+, tomllib is part of the standard library. tomli-w (TOML writer) is
kept as it's still needed for writing TOML files.

Changes:
- Removed build/pkgs/tomli directory
- Updated package dependencies to remove tomli references
- Remove the unused 3.10 env file

<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->



### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#41036
Reported by: Chenxin Zhong
Reviewer(s): Copilot, Dima Pasechnik, Tobias Diez
vbraun pushed a commit to vbraun/sage that referenced this pull request Oct 20, 2025
sagemathgh-41036: Remove tomli
    
tomli is a TOML parser that was needed for Python < 3.11. Since Python
3.11+, tomllib is part of the standard library. tomli-w (TOML writer) is
kept as it's still needed for writing TOML files.

Changes:
- Removed build/pkgs/tomli directory
- Updated package dependencies to remove tomli references
- Remove the unused 3.10 env file

<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->



### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#41036
Reported by: Chenxin Zhong
Reviewer(s): Copilot, Dima Pasechnik, Tobias Diez
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants