[Outreachy Round 27] Create ReferenceCounterApi class to hit new Reference Counter API

[gabina]

What this PR does

This PR is part of the "Improve how Wiki Education Dashboard counts references added" project (read issue #5547).

The purpose of this PR is to add a ReferenceCounterApi class, which is in charge of making requests to the new Toolforge Reference Counter API.

The ReferenceCounterApi is quite simple. It's intended to be used to retrieve the number of references in a given revision id, across every wiki (language/project) except for wikidata (this is because wikidata works pretty different from other wikis, and there is a specific gem to get references from it).

The main entry point for this class is get_number_of_references_from_revision_id method. It takes the revision id, and returns the number of references existing in the revision, or -1 if there was any problem hitting the API (as a side effect, it logs errors in Sentry).

This PR also adds basic specs for ReferenceCounterApi.

Open questions and concerns

For now, ReferenceCounterApi class processes one revision ID per request due to limitations in the Toolforge Reference Counter API, which doesn't support batching (querying more than one revision ID in a single request). If we we observe a decline in performance as a result of this approach, we will explore modifications to the Toolforge API to enable batch processing.

[gabina]

I did use the ApiErrorHandling here because I'm rescuing a real error.
We may want to add a retry strategy here, as maybe the errors in TYPICAL_ERRORS are frequent? Do you know if there is any common strategy that is used in the project?

[gabina]

@Aminehassou feel free to take a look at the PR when you have the chance!

[gabina]

CI is failing, but I don't think those errors are related to my changes.

Finished in 27 minutes 17 seconds (files took 8.55 seconds to load)
1839 examples, 3 failures, 17 pending

Failed examples:

rspec ./spec/features/timeline_editing_spec.rb:194 # timeline editing restores content for all blocks with "Discard All Changes"
rspec ./spec/lib/alerts/unsubmitted_course_alert_manager_spec.rb:56 # UnsubmittedCourseAlertManager when the alerts do not exist does not create alerts for courses that were created too recently
rspec ./spec/lib/importers/revision_score_importer_spec.rb:205 # RevisionScoreImporter.update_revision_scores_for_all_wikis imports data and calcuates an article completeness score for available wikis

There is still an error related to the browser. I don't know if it's related to the new --headless=new argument.

  2) timeline editing restores content for all blocks with "Discard All Changes"
     Got 0 failures and 2 other errors:

     2.1) Failure/Error: find(".week-1 .block-kind-#{Block::KINDS['milestone']}").hover

          Selenium::WebDriver::Error::MoveTargetOutOfBoundsError:
            move target out of bounds
              (Session info: chrome=120.0.6099.109)
          # #0 0x55b1483ccd33 <unknown>
          # #1 0x55b148089dbb <unknown>
          # #2 0x55b14812e799 <unknown>
          # #3 0x55b148103342 <unknown>
          # #4 0x55b148122297 <unknown>
          # #5 0x55b1481030e3 <unknown>
          # #6 0x55b1480cb044 <unknown>
          # #7 0x55b1480cc44e <unknown>
          # #8 0x55b148391861 <unknown>
          # #9 0x55b148395785 <unknown>
          # #10 0x55b14837f285 <unknown>
          # #11 0x55b14839641f <unknown>
          # #12 0x55b14836320f <unknown>
          # #13 0x55b1483ba028 <unknown>
          # #14 0x55b1483ba1f7 <unknown>
          # #15 0x55b1483cbed4 <unknown>
          # #16 0x7f0950294ac3 <unknown>
          # ./spec/features/timeline_editing_spec.rb:207:in `block (2 levels) in <top (required)>'
          # ------------------
          # --- Caused by: ---
          # Selenium::WebDriver::Error::MoveTargetOutOfBoundsError:
          #   move target out of bounds
          #     (Session info: chrome=120.0.6099.109)
          #   #0 0x55b1483ccd33 <unknown>

     2.2) Failure/Error: Capybara::Screenshot.new.screenshot_and_save_page if example.exception

          NoMethodError:
            undefined method `new' for Capybara::Screenshot:Module

                Capybara::Screenshot.new.screenshot_and_save_page if example.exception
                                    ^^^^
          # ./spec/rails_helper.rb:101:in `block (2 levels) in <top (required)>'

[ragesoss]

I think I've fixed all the unrelated spec failures. If you rebase this, hopefully it will be passing now.

[gabina]

@ragesoss you can take a look at this whenever you have the chance, CI is green now!

[ragesoss]

This looks solid overall!

[ragesoss]

Why return -1 (rather than nil, for example)? This class is mainly dealing with integers that represent reference counts, so using an integer as an error/warning signal is likely to lead to confusion at some point.

[gabina]

Initially, I considered using -1 as a means to distinguish (at the database level) between revisions where there was an error attempting to retrieve the number of references, and revisions for which we have not yet calculated that.
However, this approach was based on the assumption that the references field in the database would be an integer, which is not the case (we'll be using the existing features field). I believe I can find an alternative method to distinguish between these two cases after refactoring the revision score importer.
I changed this to return nil instead in ca7a4b2
Edit: the good commit is c2bf974

[Aminehassou]

Changes look good. Good work!

[gabina]

Followed the same approach as for the LiftWingApi: errors are accumulated and logged at the batch level.
However, warnings related to responses not being 200 OK keep being logged individually. I think that makes sense for now, because the reference-counter API is under our own control, so we have to monitor this to be able to fix all the necessary bugs. If at some point we find benign frequent non-200 responses and we don't want to log all of them, we can easily change this.