Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency activity collection #1609

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open

Dependency activity collection #1609

wants to merge 29 commits into from

Conversation

andrew
Copy link
Contributor

@andrew andrew commented Jul 19, 2017

First attempt at collecting and storing dependency activity in the database for #1603 and ultimately https://github.com/librariesio/supporters/issues/21

Adds a method to clone a repository, analyse its history (using repo_miner) and then store each action as a DependencyActivity record, including finding the project id by it's name in a process (if possible)

Includes a few indexes on the table but not 100% sure what we're going to want to do with the data yet so keeping them to a minimum for now.

Partially blocked by librariesio/repo_miner#2 right now

* master: (26 commits)
  Add concurrent-ruby-ext gem to speed up sidekiq
  Update appsignal
  Updated capistrano version lock
  Updated rails to 5.0.6
  Update dependencies
  Update README.md
  Update docs to reflect that watcher is now dispatch
  Fix typo
  Update pictogram with latest puppet logo
  Update counter_culture, fixes #1690
  Update dependencies
  Show all licenses in some places, fixes #1678
  Improve docs around running a background worker
  Handle download_members being called on github users gracefully
  bumb gitlab API version (#1684)
  Update dependencies
  Don't strip booleans
  Update brakeman
  Support markdown readmes for gitlab and bitbucket
  Fix double escaping issues with gitlab api
  ...
…o/libraries.io into dependency_activities

* 'dependency_activities' of https://github.com/librariesio/libraries.io:
  Skip mining dependency activities from forks
  Update appsignal
  Import dependency activities in one sql query using activerecord-import
  Only clone the default branch
  Projects have dependency activities
  Update bibliothecary and repo_miner
  Better error handling
  Fix typo
  Skip previously mined commits
  Save mined activities to the database
  Update repo_miner
  WIP dependency activity collection
* master: (114 commits)
  Handle repos with nil avatars
  Update bugsnag
  Update button copy for subscribed projects (#1827)
  Revert "Disable test for issue/pull request events from GH" (#1837)
  Stub jam package manager project method
  Show warning in header when in read-only mode
  Cap command to restart only web servers
  Rollback premailer-rails update
  Disable rake tasks when in READ_ONLY mode
  Update dependencies
  Change enable_private link in notice email
  Hook up opt-in link in notice email
  Notice email content and mailer preview
  Update terms
  Update privacy policy
  Add footer hr back in
  Swap png for svg
  Scaffolding for Tidelift notice email
  Replace BNS logo in footer with Tidelift
  Remove safestack logo
  ...
@andrew
Copy link
Contributor Author

andrew commented Nov 24, 2017

@BenJam gunna hold off on merging this until the open data release prep is done, mostly because this will put an unknown amount of load on the queue 🙈

@BenJam
Copy link
Contributor

BenJam commented Nov 24, 2017

👍

@andrew andrew removed their assignment Feb 27, 2018
@andrew andrew modified the milestones: July 2017, Next Feb 27, 2018
Base automatically changed from master to main February 4, 2021 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants