Integrate Github API for Projects and map the API data to a unified Project schema.

[kreeves]

Pull from Github the following information and make it simple to conform with final front-end JSON model.

• Grab project summary (title, project name, languages)
  • https://developer.github.com/v3/repos/#get
• Get project issues (title, labels)
  • https://developer.github.com/v3/issues/#list-issues-for-a-repository

[kreeves]

Perhaps we could open up the table for a bit of discussion about the API usage. I do have one concern related to it. We may run into an issue with the number of API calls that are required per page load.

For each project, we'll need to grab it's initial project information as well as all of it's issues. That's two calls to the API for each project. From what I can gather, there's a limit of 5000 calls to the API. At some time interval, that number gets refreshed.

There is a search API available. https://developer.github.com/v3/search/ I'm trying to see if we can somehow leverage that to limit the number of API calls.

I imagine it's also possible this could be an issue with the CFA API.

Thoughts? Am I over-thinking this concern? :)

[rlh-aagis]

I agree with the need to discuss as I too have concerns - they arose from
pulling the initial brigade location data for the map using the CfA API,
although I admit I am still researching this from a Leaflet Map API
perspective...

Paul, can we discuss this next meeting?
On Apr 7, 2015 9:02 AM, "kreeves" [email protected] wrote:

Perhaps we could open up the table for a bit of discussion about the API
usage. I do have one concern related to it. We may run into an issue with
the number of API calls that are required per page load.

For each project, we'll need to grab it's initial project information as
well as all of it's issues. That's two calls to the API for each project.
From what I can gather, there's a limit of 5000 calls to the API. At some
time interval, that number gets refreshed.

There is a search API available. https://developer.github.com/v3/search/
I'm trying to see if we can somehow leverage that to limit the number of
API calls.

I imagine it's also possible this could be an issue with the CFA API.

Thoughts? Am I over-thinking this concern? :)

—
Reply to this email directly or view it on GitHub
#61 (comment)
.

[jstirnaman]

Angular's $http service has built-in caching. We haven't configured it yet in development. Helpful overview: https://www.ng-book.com/p/Caching/ I think this will partially address the concerns.

[jstirnaman]

ProjectsService now pulls data from CFA and Github. Merging is simplistic, but it's a start. Still need to come up with a final attribute mapping/schema and update the views to match. Commit 911fc20

[jstirnaman]

The Github API returns alot of data for repos. Can we be more selective with the API?

[jstirnaman]

Closed by mistake.

[jstirnaman]

I don't see any way in the documentation to select only a subset of attributes from the API response. What is returned for /repos is already a subset of repo details.

[jstirnaman]

We can get contributors and issues as separate API calls if those are all we need from Github. The downside is that makes our app a little more complex.

[jstirnaman]

@aertoria @zmon I think we'll need to fetch issues for each Github repo (Project) through a separate API call. Since the Google Spreadsheet is the authority, it may be best to only fetch issues and contributors through separate API calls for each project in the sheet. However, that's a fairly loose linkage between the URLs in the spreadsheet and CFA API and the Github API URLs.

[jstirnaman]

@zmon @tangospring Is it enough to fetch all the labels for each project repo, i.e. using https://developer.github.com/v3/issues/labels/#list-all-labels-for-this-repository? Do we need any more info than that?

[jstirnaman]

Nevermind. I see from #17 that we'll need to return labels for only open issues. I think that means we'll have to use the /issues/ endpoint anyway.