Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

On many Concurent Request the TLM Server sometimes crashes #247

Open
wp99cp opened this issue Nov 6, 2023 · 1 comment
Open

On many Concurent Request the TLM Server sometimes crashes #247

wp99cp opened this issue Nov 6, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@wp99cp
Copy link
Member

wp99cp commented Nov 6, 2023 •
edited
Loading

Error Log:

automatic_walk-time_tables-awt-swiss-tml-api-1         | [ERROR] Error in "Index_NearestNeighbors_obj": IllegalStateException: readNode: failed reading the correct node type information
automatic_walk-time_tables-awt-swiss-tml-api-1         | [ERROR] Exception on /swiss_name [GET]
automatic_walk-time_tables-awt-swiss-tml-api-1         | Traceback (most recent call last):
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
automatic_walk-time_tables-awt-swiss-tml-api-1         |     response = self.full_dispatch_request()
automatic_walk-time_tables-awt-swiss-tml-api-1         |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |     rv = self.handle_user_exception(e)
automatic_walk-time_tables-awt-swiss-tml-api-1         |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask_cors/extension.py", line 165, in wrapped_function
automatic_walk-time_tables-awt-swiss-tml-api-1         |     return cors_after_request(app.make_response(f(*args, **kwargs)))
automatic_walk-time_tables-awt-swiss-tml-api-1         |                                                 ^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |     rv = self.dispatch_request()
automatic_walk-time_tables-awt-swiss-tml-api-1         |          ^^^^^^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
automatic_walk-time_tables-awt-swiss-tml-api-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/app.py", line 91, in get_name
automatic_walk-time_tables-awt-swiss-tml-api-1         |     raise e
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/app.py", line 62, in get_name
automatic_walk-time_tables-awt-swiss-tml-api-1         |     swiss_names = name_index.get_names(lat, lon, 3)
automatic_walk-time_tables-awt-swiss-tml-api-1         |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/swiss_TML_api/name_finding/name_finder.py", line 43, in get_names
automatic_walk-time_tables-awt-swiss-tml-api-1         |     self.index.nearest((lat, lon, lat, lon), num_results=n, objects="raw")
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/index.py", line 1028, in nearest
automatic_walk-time_tables-awt-swiss-tml-api-1         |     return self._nearest_obj(coordinates, num_results, objects)
automatic_walk-time_tables-awt-swiss-tml-api-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/index.py", line 956, in _nearest_obj
automatic_walk-time_tables-awt-swiss-tml-api-1         |     core.rt.Index_NearestNeighbors_obj(
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/core.py", line 13, in check_return
automatic_walk-time_tables-awt-swiss-tml-api-1         |     raise RTreeError(msg)
automatic_walk-time_tables-awt-swiss-tml-api-1         | rtree.exceptions.RTreeError: Error in "Index_NearestNeighbors_obj": IllegalStateException: readNode: failed reading the correct node type information

and sometimes you get:

automatic_walk-time_tables-awt-swiss-tml-api-1         | [2023-11-06 19:51:55 +0000] [7] [WARNING] Worker with pid 355 was terminated due to signal 11

automatic_walk-time_tables-awt-swiss-tml-api-1         | Fatal Python error: Segmentation fault
automatic_walk-time_tables-awt-swiss-tml-api-1         | 
automatic_walk-time_tables-awt-swiss-tml-api-1         | Thread 0x00007fd697fff700 (most recent call first):
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/index.py", line 956 in _nearest_obj
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/index.py", line 1028 in nearest
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/swiss_TML_api/name_finding/name_finder.py", line 43 in get_names
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/app.py", line 65 in get_name
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469 in dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484 in full_dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190 in wsgi_app
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2213 in __call__
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 323 in handle_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 271 in handle
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 975 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap
automatic_walk-time_tables-awt-swiss-tml-api-1         | 
automatic_walk-time_tables-awt-swiss-tml-api-1         | Current thread 0x00007fd69cf20700 (most recent call first):
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/index.py", line 956 in _nearest_obj
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/rtree/index.py", line 1028 in nearest
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/swiss_TML_api/name_finding/name_finder.py", line 43 in get_names
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/app/app.py", line 65 in get_name
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469 in dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484 in full_dispatch_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190 in wsgi_app
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2213 in __call__
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 323 in handle_request
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 271 in handle
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 975 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap
automatic_walk-time_tables-awt-swiss-tml-api-1         | 
automatic_walk-time_tables-awt-swiss-tml-api-1         | Thread 0x00007fd69d725700 (most recent call first):
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/reloader.py", line 54 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap
automatic_walk-time_tables-awt-swiss-tml-api-1         | 
automatic_walk-time_tables-awt-swiss-tml-api-1         | Thread 0x00007fd6a2da8740 (most recent call first):
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/selectors.py", line 468 in select
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 200 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/base.py", line 142 in init_process
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 92 in init_process
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 589 in spawn_worker
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 622 in spawn_workers
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 551 in manage_workers
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 202 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/app/base.py", line 72 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/app/base.py", line 231 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 67 in run
automatic_walk-time_tables-awt-swiss-tml-api-1         |   File "/usr/local/bin/gunicorn", line 8 in <module>
@wp99cp wp99cp added the bug Something isn't working label Nov 6, 2023
@wp99cp wp99cp mentioned this issue Nov 6, 2023
Closed
@wp99cp
Copy link
Member Author

wp99cp commented Jul 2, 2024

related with #383 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@wp99cp