Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only sync past day of subscriptions instead of default 2 days to avoid timeout #155

Merged
merged 1 commit into from
Sep 16, 2024

Conversation

jterry64
Copy link
Member

Pull request checklist

Please check if your PR fulfills the following requirements:

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side). Don't request your master!
  • Make sure you are making a pull request against the develop branch (left side). Also you should start your branch off our develop.
  • Check the commit's or even all commits' message styles matches our requested structure.
  • Check your code additions will fail neither code linting checks nor unit test.

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

The call to /area/sync is timing out every night. Without date parameter, it defaults to syncing 2 days of subscriptions.

Issue Number: N/A

What is the new behavior?

Only sync one day of subscriptions to reduce chance of timeout.

Does this introduce a breaking change?

  • Yes
  • No

Other information

Copy link
Contributor

@danscales danscales left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you said that the API syncs two days by default? But in the RW docs it says it does the last week by default ( https://resource-watch.github.io/doc-api/reference.html#sync-areas ) If it really does one week by default, I wonder if you could try maybe two or three days for syncing (rather than one), just to be more likely that no areas slip through the cracks (in case datapump somehow doesn't run one night)?

@jterry64
Copy link
Member Author

I think you said that the API syncs two days by default? But in the RW docs it says it does the last week by default ( https://resource-watch.github.io/doc-api/reference.html#sync-areas ) If it really does one week by default, I wonder if you could try maybe two or three days for syncing (rather than one), just to be more likely that no areas slip through the cracks (in case datapump somehow doesn't run one night)?

If you remember from the last huddle, even though it says a week, @solomon-negusse showed us the code actually defaults to two days: https://github.com/gfw-api/gfw-area/blob/fe2ba0b314a91eb320893591a69b7a44f3dd993f/app/src/routes/api/v2/area.router.js#L632

@jterry64 jterry64 merged commit ba465d2 into develop Sep 16, 2024
3 checks passed
@jterry64 jterry64 deleted the batman/broken_sync branch September 16, 2024 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants