Releases: UCSD-Historical-Enrollment-Data/webreg_scraper
Releases · UCSD-Historical-Enrollment-Data/webreg_scraper
[0.5.1] Dependency Updates & Minor Endpoint Update
This release serves as the official v0.5.0 release (see rc1, rc2, rc3, rc4, and rc5 for full changes), since I forgot to make a release for that.
On top of what was released in v0.5.0, the following changes have been made to this project in v0.5.1:
- Updates dependencies in all Rust binary and crate projects.
- Fixes a minor issue where the /terms endpoint requires a term query parameter.
[0.5.0-rc.5] Add Course Text & Section Text Endpoint Support
See #30 for release notes.
v0.5.0-rc.4
[0.5.0-rc.4] Adjusted Delay Time and Support for Hidden Terms (#29) * feat: change login backoff time from sec -> min * fix: hidden terms aren't being recognized by scraper * feat: add constant delay for initial login after failure - also updated version to rc.4 * feat: add extra delays between requests * chore: update scraper dependencies
v0.5.0-rc.3
[0.5.0-rc.3] Login Stat Fix (#27) * fix: return JSON obj instead of str for login script stats * chore: bump version to rc.3 * style: run cargo fmt
0.5.0-rc2
[v0.5.0-rc.2] Single Cookie Scraper (#26) * feat: automatically terminate cookie server if duo auth happens after init * feat: use exponential backoff for determining delay * refactor: fix notifier to use new API, 5 min cooldown, auth support * docs: include link to nonexistent wiki * chore: update version * chore: update cargo.lock dependency versions
v0.5.0-rc1
[v0.5] Single Cookie Scraper (#25) * feat: update wrapper and rewrite scraper for single login * refactor: use tracing `log` and `error` instead of `println` * refactor: add more logging to login script functions * docs: update README with updated examples, docs * refactor: use method to set stop flag directly * chore: move scraper code to its own module * feat: rewrote/integrate basic webreg server to scraper * chore: remove `Extension` param for course_info endpoint * chore: remove unnecessary client, close_after_request builder calls * feat: add extra endpoints for cookie-related reqs * refactor: update function accessibility level * feat: add authorization support for api * chore: update ci config * chore: update dependencies to be more general + .gitignore * chore: update `webweg` to `0.9` * feat: add `description` column to sql table * chore: move both helper scripts to `helpers` directory * refactor: use workspace to separate webreg + authmanager crates * refactor: move project dirs around, create basicauth lib for both bins * feat: add more helper methods for basicauth lib * test: add basic unit tests for basic auth * style: run cargo fmt * style: run cargo clippy * refactor: update enum variant to be more clear * feat: authmanager complete * fix: compile script for webregautoin now uses curr path for proj * chore: rename gh ci config to explicitly include bin file name * style: run `cargo clippy` and `cargo fmt` * docs: add plenty of new READMEs * docs: update hyperlinks to project folders * chore: move config.example.json to webreg dir * docs: update docs to be more relevant to current project * chore: minor word correction for no cookie found * docs: modify documentation to be more clear, remove outdated docs * chore: change version to rc.1
v0.4.7: Add Timeout to Requests
All requests will now have 25 seconds to complete or else be terminated.
v0.4.5: Bump WebWeg Library Version
This release is exactly the same as v0.4.4, but bumps the webweg library version to v0.7.4.
v0.4.4: Time Stat API Fix
This PR achieves the following:
- fixed a bug where the time stat API would return the most recent requests as a string instead of an array
- fixed a bug where calling the time stat API may require you to wait a while (up to the cooldown specified for a term) instead of nearly instantaneously.
- increased number of recent requests from 100 to 1000.
v0.4.3: Add Time Stats to Scraper
In this minor release, the scraper now keeps track of how long each request to WebReg takes. This might be useful if, for example, WebReg takes an abnormally long time to process a request.