Description
Context
Among our four existing official test reporters for ruby, python, javascript, and php, there is inconsistent support for submitting test coverage reports without git and a git repository present. While we've worked with community contributors to add partial support. this has not been prioritized internally as a general test reporter feature.
There is also inconsistent support from CI providers for commit information within the CI environment. We currently check both the environment and git for information required to generate a coverage report payload.
If the CI provider checks out the source code into a detached HEAD, we can't rely on its branch
, so we also look for it in the environment via our CI mappings. Conversely, if we can't determine the git branch from the CI environment, we have to rely on git for that information.
Generating file blob IDs for source files included within the test coverage report in order to compare test coverage data with source listings is particularly more reliable when done from git. There are ways of generating file blob IDs from the file content (and most of our existing test reporters include such a fallback) but we've found such solutions to be occasionally incompatible with git's internal methods.
Related Issues & PRs
codeclimate/ruby-test-reporter#183
codeclimate/ruby-test-reporter#177
codeclimate/ruby-test-reporter#171
codeclimate/ruby-test-reporter#168
codeclimate/ruby-test-reporter#156
codeclimate/ruby-test-reporter#116
codeclimate/ruby-test-reporter#115
codeclimate/ruby-test-reporter#89
codeclimate/ruby-test-reporter#57
codeclimate/ruby-test-reporter#56
codeclimate/ruby-test-reporter#38
codeclimate/javascript-test-reporter#44
Next Steps
We should determine whether or not to support test coverage reporting without git and a git repository present. I'm personally not 100% clear of what happens when the file blob IDs mismatch. I assume we simply don't render file coverage or file coverage annotations for that file on codeclimate.com. Do we know how often these mismatches take place?
I'll hold back any thoughts on implementation until we've addressed these basic "should we" questions.