Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BACKEND] Error on task create_update_excursions_segments after db clean #345

Open
rv2931 opened this issue Dec 11, 2024 · 0 comments · May be fixed by #346
Open

[BACKEND] Error on task create_update_excursions_segments after db clean #345

rv2931 opened this issue Dec 11, 2024 · 0 comments · May be fixed by #346
Labels
backend bug Something isn't working data

Comments

@rv2931
Copy link
Collaborator

rv2931 commented Dec 11, 2024

Afin de valider le process ETL metrics j'ai paris un dump de la production retoré sur mon test et nettoyé toutes les données calculées (et ancienne tables) donc vidé fct_segment, fct_excursion, mais aussi vessel_data, vessel_position....
Ici le script de nettoyage (qui a pris 40 minutes):

delete from public.vessel_data ;
delete from public.vessel_positions ;
delete from public.vessel_voyage ;
delete from public.vessels ;
delete from public.spire_vessel_positions ;
delete from public.tasks_executions as te where te.task_name != 'load_spire_data_from_api' or te.active = False;
delete from public.rel_segment_zone ;
delete from public.fct_segment ;
delete from public.fct_excursion ;

Lorsque j'ai exécuté la commande python bloom/task/clean_positions.py && python bloom/task/create_update_excursions_segments.py j'obtiens cette erreur
J'ai le sentiment que c'est parce que la requête ne prend pas en charge le cas ou fct_segment est vide mais c'est peut-être autre chose

Traceback (most recent call last):
  File "/project/backend/bloom/infra/database/database_manager.py", line 32, in session
    yield session
  File "/project/backend/bloom/tasks/create_update_excursions_segments.py", line 416, in run
    nb_last = segment_repository.update_last_segments(session, vessels_ids)
  File "/project/backend/bloom/infra/repositories/repository_segment.py", line 295, in update_last_segments
    last_segments = session.execute(text("""SELECT DISTINCT ON (vessel_id) s.id FROM fct_segment s
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2362, in execute
    return self._execute_internal(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2256, in _execute_internal
    result = conn.execute(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1418, in execute
    return meth(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1640, in _execute_clauseelement
    ret = self._execute_context(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 941, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax error at or near ")"
LINE 3: ...                                      WHERE vessel_id in () 
                                                                     ^

[SQL: SELECT DISTINCT ON (vessel_id) s.id FROM fct_segment s
                                                JOIN fct_excursion e ON e.id = s.excursion_id
                                                WHERE vessel_id in %(vessel_ids)s 
                                                ORDER BY vessel_id, timestamp_start DESC]
[parameters: {'vessel_ids': ()}]
@rv2931 rv2931 added bug Something isn't working backend data labels Dec 11, 2024
@rv2931 rv2931 changed the title [BACKEND] [BACKEND] Error on task create_update_excursions_segments after db clean Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend bug Something isn't working data
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant