Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENG-6708][ENG-7286] improved trovesearch_denorm... #841

Merged
merged 6 commits into from
Feb 21, 2025

Conversation

aaxelb
Copy link
Contributor

@aaxelb aaxelb commented Feb 7, 2025

improvements to trovesearch_denorm index strategy:

  • skip indexing nested "identifier" text (saves 107 fields)
  • allow wild-card properties only to depth 1 (disallow cardSearchText[*.*])
  • reduce number_of_replicas from 2 to 1
  • fix: backfilling a new index shouldn't clear earlier indexes

other:

  • use select_related to avoid n+1 queries on n search results
  • stop hashing so much rendering valuesearch responses (get unique ids other ways)

@coveralls
Copy link

coveralls commented Feb 7, 2025

Coverage Status

coverage: 91.765% (+0.02%) from 91.742%
when pulling 9ae8ba9 on aaxelb:fix/denorm-optimiz
into 86d1d56 on CenterForOpenScience:develop.

@aaxelb aaxelb force-pushed the fix/denorm-optimiz branch from 4c5b016 to a9204f3 Compare February 7, 2025 21:34
@aaxelb aaxelb changed the title [wip] faster trovesearch_denorm... [wip][ENG-6708] faster trovesearch_denorm... Feb 14, 2025
@aaxelb aaxelb changed the title [wip][ENG-6708] faster trovesearch_denorm... [ENG-6708][ENG-7286] faster trovesearch_denorm... Feb 21, 2025
@aaxelb aaxelb changed the title [ENG-6708][ENG-7286] faster trovesearch_denorm... [ENG-6708][ENG-7286] improved trovesearch_denorm... Feb 21, 2025
@aaxelb aaxelb force-pushed the fix/denorm-optimiz branch from f4b67bc to bf69f1e Compare February 21, 2025 16:59
@aaxelb aaxelb force-pushed the fix/denorm-optimiz branch from bf69f1e to 9ae8ba9 Compare February 21, 2025 17:40
@aaxelb aaxelb marked this pull request as ready for review February 21, 2025 17:46
@felliott
Copy link
Member

LGTM!

@aaxelb aaxelb merged commit c09e4f4 into CenterForOpenScience:develop Feb 21, 2025
3 checks passed
@aaxelb aaxelb deleted the fix/denorm-optimiz branch February 21, 2025 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants