Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

無法安裝和執行 LTP server #3

Open
danny0838 opened this issue May 30, 2024 · 3 comments
Open

無法安裝和執行 LTP server #3

danny0838 opened this issue May 30, 2024 · 3 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@danny0838
Copy link

系統： Windows 10, Python 3.12.3

已安裝以下套件:

fire                      0.6.0
pip                       24.0
torch                     2.3.0
tornado                   6.4

無法按照 README 提供的指令安裝 LTP；

D:\*****\test\python\ltp>pip install --force-reinstall "ltp==4.1.5.post2"
Collecting ltp==4.1.5.post2
  Using cached ltp-4.1.5.post2-py3-none-any.whl.metadata (4.8 kB)
Collecting torch>=1.2.0 (from ltp==4.1.5.post2)
  Using cached torch-2.3.0-cp312-cp312-win_amd64.whl.metadata (26 kB)
Collecting transformers<=4.7.0,>=4.0.0 (from ltp==4.1.5.post2)
  Using cached transformers-4.7.0-py3-none-any.whl.metadata (48 kB)
Collecting pygtrie<2.5,>=2.3.0 (from ltp==4.1.5.post2)
  Using cached pygtrie-2.4.2-py3-none-any.whl
Collecting packaging>=20.0 (from ltp==4.1.5.post2)
  Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB)
Collecting filelock (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB)
Collecting typing-extensions>=4.8.0 (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached typing_extensions-4.12.0-py3-none-any.whl.metadata (3.0 kB)
Collecting sympy (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached networkx-3.3-py3-none-any.whl.metadata (5.1 kB)
Collecting jinja2 (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting fsspec (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached fsspec-2024.5.0-py3-none-any.whl.metadata (11 kB)
Collecting mkl<=2021.4.0,>=2021.1.1 (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached mkl-2021.4.0-py2.py3-none-win_amd64.whl.metadata (1.4 kB)
Collecting huggingface-hub==0.0.8 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached huggingface_hub-0.0.8-py3-none-any.whl.metadata (8.7 kB)
Collecting numpy>=1.17 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl.metadata (61 kB)
Collecting pyyaml (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached PyYAML-6.0.1-cp312-cp312-win_amd64.whl.metadata (2.1 kB)
Collecting regex!=2019.12.17 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached regex-2024.5.15-cp312-cp312-win_amd64.whl.metadata (41 kB)
Collecting requests (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting sacremoses (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached sacremoses-0.1.1-py3-none-any.whl.metadata (8.3 kB)
Collecting tokenizers<0.11,>=0.10.1 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached tokenizers-0.10.3.tar.gz (212 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting tqdm>=4.27 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
Collecting intel-openmp==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl.metadata (1.2 kB)
Collecting tbb==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached tbb-2021.12.0-py3-none-win_amd64.whl.metadata (1.1 kB)
Collecting colorama (from tqdm>=4.27->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl.metadata (3.1 kB)
Collecting charset-normalizer<4,>=2 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
Collecting click (from sacremoses->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting joblib (from sacremoses->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB)
Collecting mpmath<1.4.0,>=1.1.0 (from sympy->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Using cached ltp-4.1.5.post2-py3-none-any.whl (94 kB)
Using cached packaging-24.0-py3-none-any.whl (53 kB)
Using cached torch-2.3.0-cp312-cp312-win_amd64.whl (159.7 MB)
Using cached transformers-4.7.0-py3-none-any.whl (2.5 MB)
Using cached huggingface_hub-0.0.8-py3-none-any.whl (34 kB)
Using cached mkl-2021.4.0-py2.py3-none-win_amd64.whl (228.5 MB)
Using cached intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl (3.5 MB)
Using cached tbb-2021.12.0-py3-none-win_amd64.whl (286 kB)
Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl (15.5 MB)
Using cached regex-2024.5.15-cp312-cp312-win_amd64.whl (268 kB)
Using cached tqdm-4.66.4-py3-none-any.whl (78 kB)
Using cached typing_extensions-4.12.0-py3-none-any.whl (37 kB)
Using cached filelock-3.14.0-py3-none-any.whl (12 kB)
Using cached fsspec-2024.5.0-py3-none-any.whl (316 kB)
Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Using cached networkx-3.3-py3-none-any.whl (1.7 MB)
Using cached PyYAML-6.0.1-cp312-cp312-win_amd64.whl (138 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached sacremoses-0.1.1-py3-none-any.whl (897 kB)
Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB)
Using cached certifi-2024.2.2-py3-none-any.whl (163 kB)
Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl (100 kB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl (17 kB)
Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
Using cached urllib3-2.2.1-py3-none-any.whl (121 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Using cached joblib-1.4.2-py3-none-any.whl (301 kB)
Building wheels for collected packages: tokenizers
  Building wheel for tokenizers (pyproject.toml): started
  Building wheel for tokenizers (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  Building wheel for tokenizers (pyproject.toml) did not run successfully.
  exit code: 1
  
  [51 lines of output]
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-cpython-312
  creating build\lib.win-amd64-cpython-312\tokenizers
  copying py_src\tokenizers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers
  creating build\lib.win-amd64-cpython-312\tokenizers\models
  copying py_src\tokenizers\models\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\models
  creating build\lib.win-amd64-cpython-312\tokenizers\decoders
  copying py_src\tokenizers\decoders\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\decoders
  creating build\lib.win-amd64-cpython-312\tokenizers\normalizers
  copying py_src\tokenizers\normalizers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\normalizers
  creating build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers
  copying py_src\tokenizers\pre_tokenizers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers
  creating build\lib.win-amd64-cpython-312\tokenizers\processors
  copying py_src\tokenizers\processors\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\processors
  creating build\lib.win-amd64-cpython-312\tokenizers\trainers
  copying py_src\tokenizers\trainers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\trainers
  creating build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\base_tokenizer.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\bert_wordpiece.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\byte_level_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\char_level_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\sentencepiece_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\sentencepiece_unigram.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  creating build\lib.win-amd64-cpython-312\tokenizers\tools
  copying py_src\tokenizers\tools\visualizer.py -> build\lib.win-amd64-cpython-312\tokenizers\tools
  copying py_src\tokenizers\tools\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\tools
  copying py_src\tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers
  copying py_src\tokenizers\models\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\models
  copying py_src\tokenizers\decoders\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\decoders
  copying py_src\tokenizers\normalizers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\normalizers
  copying py_src\tokenizers\pre_tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers
  copying py_src\tokenizers\processors\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\processors
  copying py_src\tokenizers\trainers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\trainers
  copying py_src\tokenizers\tools\visualizer-styles.css -> build\lib.win-amd64-cpython-312\tokenizers\tools
  running build_ext
  running build_rust
  error: can't find Rust compiler
  
  If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
  
  To update pip, run:
  
      pip install --upgrade pip
  
  and then retry package installation.
  
  If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
  [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

如果用其他方式安裝以下套件：

ltp                       4.2.13
ltp-core                  0.1.4
ltp-extension             0.1.13

執行 LTP server 時仍會出錯：

D:\*****\test\python\ltp>python server.base.py serve
C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Traceback (most recent call last):
  File "D:\*****\test\python\ltp\server.base.py", line 154, in <module>
    Fire(Server)
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\fire\core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\fire\core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\fire\core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "D:\*****\test\python\ltp\server.base.py", line 52, in __init__
    self.ltp = LTP(path=path, device=device)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\ltp\interface.py", line 125, in LTP
    return LTP_neural._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\ltp\nerual.py", line 534, in _from_pretrained
    ltp = cls(**model_kwargs).to(map_location)
          ^^^^^^^^^^^^^^^^^^^
TypeError: LTP.__init__() got an unexpected keyword argument 'path'
@kanasimi kanasimi added help wanted Extra attention is needed enhancement New feature or request labels May 31, 2024
@kanasimi
Copy link
Owner

kanasimi commented Jun 1, 2024

有些技術問題需解決
HIT-SCIR/ltp#699

@danny0838
Copy link
Author

你貼的連結和頂樓說的出錯問題似乎沒有關係？

@kanasimi
Copy link
Owner

kanasimi commented Jun 1, 2024 •
edited
Loading

您或許可安裝最新版(ltp 4.2.13)後，嘗試用這個方法修復
HIT-SCIR/ltp#701

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kanasimi @danny0838