Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Data7 configuration #324

Merged
merged 3 commits into from
Jan 8, 2025
Merged

Update Data7 configuration #324

merged 3 commits into from
Jan 8, 2025

Conversation

jmaupetit
Copy link
Member

Purpose

Now that we have production-ready data, we need to carefully configure data7 given its usage.

Proposal

  • run app using a single worker
  • upgrade data7 to 0.9.0
  • update latest statuses query

@jmaupetit jmaupetit self-assigned this Jan 8, 2025
@jmaupetit jmaupetit added improvement needs review opendata Item related to the opendata service labels Jan 8, 2025
@jmaupetit jmaupetit force-pushed the fix-data7-uvicorn-boot branch from 240200f to 294d093 Compare January 8, 2025 09:11
Copy link
Collaborator

@loco-philippe loco-philippe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok pour la requete,

on peut éviter une jointure avec :

WITH
  pdc_status AS (
    SELECT
      LAST (Status.id, Status.horodatage) AS latest_id,
      id_pdc_itinerance
    FROM
      PointDeCharge
      INNER JOIN Status ON Status.point_de_charge_id = PointDeCharge.id
    GROUP BY
      PointDeCharge.id_pdc_itinerance
  )
SELECT
  -- PointDeCharge.id_pdc_itinerance,
  Status.etat_pdc,
  Status.occupation_pdc,
  Status.horodatage,
  Status.etat_prise_type_2,
  Status.etat_prise_type_combo_ccs,
  Status.etat_prise_type_chademo,
  Status.etat_prise_type_ef
FROM
  Status
  -- INNER JOIN PointDeCharge ON Status.point_de_charge_id = PointDeCharge.id
WHERE
  Status.id IN (
    SELECT
      latest_id
    FROM
      pdc_status
  )
ORDER BY
  Status.horodatage DESC

@loco-philippe
Copy link
Collaborator

Non,c'est pas bon du tout. J'ai été trop vite!!!

@jmaupetit jmaupetit force-pushed the fix-data7-uvicorn-boot branch from 9b231f8 to 294d093 Compare January 8, 2025 18:01
As the opendata application is not supposed to get a high traffic
(thanks to the NAP), a single worker with minimal configuration would be
sufficient. This will avoid consuming too much RAM.
This new release upgrade dependencies and allows to configure Sentry's
profiling.
Now that we have real world statuses, we can provide a relevant
timescaleDB based database query.
@jmaupetit jmaupetit force-pushed the fix-data7-uvicorn-boot branch from 294d093 to 8e3ffcb Compare January 8, 2025 18:02
Copy link

github-actions bot commented Jan 8, 2025

Current benchmark

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% git timestamp version
GET /auth/whoami 7 0 12.0 17.751331571430132 6.648925999996891 56.539098000001786 160.0 0.8600524178572233 0.0 12 13 16 16 57 57 57 57 57 57 57 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
POST /dynamique/session/ 7 0 22.0 22.558368999999143 10.901028999995788 40.36439600000108 45.0 0.8600524178572233 0.0 22 25 33 33 40 40 40 40 40 40 40 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
POST /dynamique/session/bulk 6 0 60.0 506.32648850000334 27.555475000013985 1607.6605050000126 411.0 0.7371877867347628 0.0 640 640 650 650 1600 1600 1600 1600 1600 1600 1600 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
GET /dynamique/status/ 9 0 24.0 328.65335766666703 9.964362000005169 1453.242423000006 285.0 1.1057816801021445 0.0 24 76 650 680 1500 1500 1500 1500 1500 1500 1500 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
POST /dynamique/status/ 7 0 41.0 157.70081971428291 22.73138900000049 646.6131829999995 45.0 0.8600524178572233 0.0 41 47 290 290 650 650 650 650 650 650 650 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3 5 0 18.0 152.83685259998947 6.810349999994969 698.884755999984 283.0 0.6143231556123023 0.0 18 26 26 700 700 700 700 700 700 700 700 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3/history 5 0 59.0 93.6387582000009 23.53282999999351 293.345525999996 17211.4 0.6143231556123023 0.0 59 59 59 290 290 290 290 290 290 290 290 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
POST /dynamique/status/bulk 8 0 20.0 158.39576925000287 13.84292599999526 453.7638040000047 411.0 0.9829170489796838 0.0 27 310 410 410 450 450 450 450 450 450 450 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
POST /statique/ 3 0 76.0 265.5152656666739 32.181203000021696 688.6022219999859 68.0 0.3685938933673814 0.0 76 76 690 690 690 690 690 690 690 690 690 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
GET /statique/?limit=100 1 0 181.99740899999028 181.99740899999028 181.99740899999028 181.99740899999028 129006.0 0.1228646311224604 0.0 180 180 180 180 180 180 180 180 180 180 180 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
PUT /statique/FR073EP7QU5233 6 0 76.0 231.96157166665895 24.21419799998148 637.0405210000172 1371.0 0.7371877867347628 0.0 180 180 430 430 640 640 640 640 640 640 640 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
GET /statique/FRALLEGO002006P3 5 0 47.0 309.5055679999973 22.10061099998484 801.7829380000023 1202.0 0.6143231556123023 0.0 47 650 650 800 800 800 800 800 800 800 800 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
POST /statique/bulk 9 0 760.0 944.045043666664 416.52622499998415 1608.9372079999862 1113.3333333333333 1.1057816801021445 0.0 760 1100 1200 1600 1600 1600 1600 1600 1600 1600 1600 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0
Aggregated 78 0 44.0 285.84234447435773 6.648925999996891 1608.9372079999862 3218.038461538461 9.583441227551916 0.0 47 290 450 650 760 1500 1600 1600 1600 1600 1600 d660acc 2025-01-08 18:03:08.081692+00:00 0.16.0

Comparison with the latest previous benchmark

A lower (negative) value means the current version performs better than the previous one.

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s
GET /auth/whoami
POST /dynamique/session/
POST /dynamique/session/bulk
GET /dynamique/status/
POST /dynamique/status/
GET /dynamique/status/FRALLEGO002006P3
GET /dynamique/status/FRALLEGO002006P3/history
POST /dynamique/status/bulk
POST /statique/
GET /statique/?limit=10
GET /statique/?limit=100
PUT /statique/FR073EP7QU5233
GET /statique/FRALLEGO002006P3
POST /statique/bulk
Aggregated

@jmaupetit jmaupetit merged commit 5fd721d into main Jan 8, 2025
7 checks passed
@jmaupetit jmaupetit deleted the fix-data7-uvicorn-boot branch January 8, 2025 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement needs review opendata Item related to the opendata service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants