Fixes 500 links loading issue #5662
Conversation
|
@ragesoss I realized the issue in the previous PR as I was extracting continue parameter from response.data whereas it should have been extracted from response only. I believe now the modules are being loaded fine. Please review the PR. |
| # To handle more than 500 pages linked from the source page, | ||
| # we'll need to update this to use 'continue'. |
There was a problem hiding this comment.
If this PR implements the use of 'continue' then you should remove/update this comment
| # we'll need to update this to use 'continue'. | ||
| query_params = { prop: 'links', titles: @wiki_base_page, pllimit: 500 } | ||
| response = WikiApi.new(MetaWiki.new).query(query_params) | ||
| @query.merge! @continue unless @continue.nil? |
There was a problem hiding this comment.
What is the sense of this line? If @continue is not previously defined then it will be always nil?
There was a problem hiding this comment.
@fillingtothemomo my previous comment was in relation to the following line:
@query.merge! @continue unless @continue.nil?
what I meant is that if @continue is not previously defined, then that line has no effect.
There was a problem hiding this comment.
Could you please explain me what the following line does?
@query.merge! @continue unless @continue.nil?
There was a problem hiding this comment.
It is supposed to be merging the @continue content into the query unless it is null.
There was a problem hiding this comment.
And what is the content of @continue?
There was a problem hiding this comment.
It's nil currently. I tried with @continue = response['continue'] as well so it wouldn't be null, but didn't succeed
There was a problem hiding this comment.
Ok, you have to fix that. What the answer looks like? That should help you understand why @continue = response['continue'] doesn't work.
fixes #5588
What this PR does
Implements continue in wiki_source_pages method in
lib/training/wiki_training_loader.rbwhich is responsible for fetching the links and earlier it was only fetching 500 links and now it fetches till continue parameter becomes nil.Screenshots
Before:
After: