Fix part of #5343: Generate Code Coverage Report in HTML and MARKDOWN formats #5443

Rd4dev · 2024-06-24T13:14:50Z

Explanation

Fixes part of #5343

Project

[PR 1.5 of Project 4.1]

Changes Made

This PR introduces the CoverageReporter utility to generate the code coverage report. The main features and changes include:

New Utility:
- CoverageReporter to generate code coverage reports.
Command Line Arguments:
- Now support output formats: HTML or MARKDOWN.
Report Generation:
- Generates coverage reports from the generated proto for the requested format.
- Outputs are generated and stored in the coverage_reports folder relative to the repoRoot.
Gitignore Update:
- The coverage_reports folder is added to .gitignore to ensure it is not added to the repository.

Essential Checklist

The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".)
Any changes to scripts/assets files have their rationale included in the PR explanation.
The PR follows the style guide.
The PR does not contain any unnecessary code changes from Android Studio (reference).
The PR is made from a branch that's not called "develop" and is up-to-date with "develop".
The PR is assigned to the appropriate reviewers (reference).

For UI-specific PRs only

If your PR includes UI-related changes, then:

Add screenshots for portrait/landscape for both a tablet & phone of the before & after UI changes
For the screenshots above, include both English and pseudo-localized (RTL) screenshots (see RTL guide)
Add a video showing the full UX flow with a screen reader enabled (see accessibility guide)
For PRs introducing new UI elements or color changes, both light and dark mode screenshots must be included
Add a screenshot demonstrating that you ran affected Espresso tests locally & that they're passing

… data file path

… file content and its related test content in the provided subpackage

…d test files with corresponding build configurations and content

…the contents as byte array

…nary

…eturning coverage data

… arguments and use them

… methods

…e in testing the utilities

…o code_coverage_bazel_command_execution

…instead of test target

Rd4dev · 2024-06-27T06:13:37Z

@adhiamboperes, the PR 1.5 tasks are done.

This now introduces the CoverageReporter that outputs the coverage data in .md and .html formats as represented in the Fix part of #5343: Generate Code Coverage Report in HTML and MARKDOWN formats #5443 (comment)
Also updated the tests to check for (app, scripts and default modules) in both (html and md formats)

As of now the reports are returning a pair of computed coverage ratio and the actual report text.
And RunCoverage takes the report text and saves it to the output file (in location coverage_reports/.../coverage.md or .html)

M2
These report texts and computed coverage ratios are returned with the plan to make them useful in M2

report text - to gather all the reports (failed) and upload them as a single comment
computed coverage ratio - to check for minimum threshold to decide pass / fail case

with these plans, I have addressed the existing review comments, would it be possible to PTAL!

adhiamboperes

@Rd4dev, PTAL at some comments that I have.

scripts/src/javatests/org/oppia/android/scripts/coverage/CoverageReporterTest.kt

scripts/src/javatests/org/oppia/android/scripts/coverage/RunCoverageTest.kt

scripts/src/java/org/oppia/android/scripts/coverage/RunCoverage.kt

scripts/src/java/org/oppia/android/scripts/coverage/CoverageReporter.kt

oppiabot · 2024-06-27T18:30:20Z

Unassigning @adhiamboperes since the review is done.

adhiamboperes

A couple more followups

scripts/src/java/org/oppia/android/scripts/coverage/RunCoverage.kt

scripts/src/java/org/oppia/android/scripts/coverage/CoverageReporter.kt

github-actions · 2024-06-28T02:53:30Z

APK & AAB differences analysis

Note that this is a summarized snapshot. See the CI artifacts for detailed differences.

Dev

Expand to see flavor specifics

Universal APK

APK file size: 16 MiB (old), 16 MiB (new), 0 bytes (No change)

APK download size (estimated): 14 MiB (old), 14 MiB (new), 11 bytes (Added)

Method count: 227007 (old), 227007 (new), 0 (No change)

Features: 2 (old), 2 (new), 0 (No change)

Permissions: 6 (old), 6 (new), 0 (No change)

Resources: 6550 (old), 6550 (new), 0 (No change)

Anim: 49 (old), 49 (new), 0 (No change)
Animator: 20 (old), 20 (new), 0 (No change)
Array: 15 (old), 15 (new), 0 (No change)
Attr: 915 (old), 915 (new), 0 (No change)
Bool: 10 (old), 10 (new), 0 (No change)
Color: 911 (old), 911 (new), 0 (No change)
Dimen: 994 (old), 994 (new), 0 (No change)
Drawable: 373 (old), 373 (new), 0 (No change)
Id: 1206 (old), 1206 (new), 0 (No change)
Integer: 37 (old), 37 (new), 0 (No change)
Interpolator: 11 (old), 11 (new), 0 (No change)
Layout: 368 (old), 368 (new), 0 (No change)
Menu: 1 (old), 1 (new), 0 (No change)
Mipmap: 1 (old), 1 (new), 0 (No change)
Plurals: 10 (old), 10 (new), 0 (No change)
Raw: 2 (old), 2 (new), 0 (No change)
String: 805 (old), 805 (new), 0 (No change)
Style: 816 (old), 816 (new), 0 (No change)
Xml: 6 (old), 6 (new), 0 (No change)

Lesson assets: 111 (old), 111 (new), 0 (No change)

AAB differences

Expand to see AAB specifics

Supported configurations:

hdpi (same)
ldpi (same)
mdpi (same)
tvdpi (same)
xhdpi (same)
xxhdpi (same)
xxxhdpi (same)

Base APK

APK file size: 16 MiB (old), 16 MiB (new), 4 bytes (Removed)
APK download size (estimated): 14 MiB (old), 14 MiB (new), 42 bytes (Removed)

Configuration hdpi

APK file size: 59 KiB (old), 59 KiB (new), 0 bytes (No change)
APK download size (estimated): 23 KiB (old), 23 KiB (new), 0 bytes (No change)

Configuration ldpi

APK file size: 56 KiB (old), 56 KiB (new), 0 bytes (No change)
APK download size (estimated): 18 KiB (old), 18 KiB (new), 0 bytes (No change)

Configuration mdpi

APK file size: 53 KiB (old), 53 KiB (new), 0 bytes (No change)
APK download size (estimated): 18 KiB (old), 18 KiB (new), 0 bytes (No change)

Configuration tvdpi

APK file size: 102 KiB (old), 102 KiB (new), 0 bytes (No change)
APK download size (estimated): 38 KiB (old), 38 KiB (new), 0 bytes (No change)

Configuration xhdpi

APK file size: 67 KiB (old), 67 KiB (new), 0 bytes (No change)
APK download size (estimated): 28 KiB (old), 28 KiB (new), 0 bytes (No change)

Configuration xxhdpi

APK file size: 76 KiB (old), 76 KiB (new), 0 bytes (No change)
APK download size (estimated): 38 KiB (old), 38 KiB (new), 0 bytes (No change)

Configuration xxxhdpi

APK file size: 79 KiB (old), 79 KiB (new), 0 bytes (No change)
APK download size (estimated): 39 KiB (old), 39 KiB (new), 0 bytes (No change)

Alpha

Expand to see flavor specifics

Universal APK

APK file size: 10 MiB (old), 10 MiB (new), 0 bytes (No change)

APK download size (estimated): 9184 KiB (old), 9184 KiB (new), 21 bytes (Removed)

Method count: 101341 (old), 101341 (new), 0 (No change)

Features: 2 (old), 2 (new), 0 (No change)

Permissions: 6 (old), 6 (new), 0 (No change)

Resources: 5504 (old), 5504 (new), 0 (No change)

Anim: 39 (old), 39 (new), 0 (No change)
Animator: 18 (old), 18 (new), 0 (No change)
Array: 14 (old), 14 (new), 0 (No change)
Attr: 879 (old), 879 (new), 0 (No change)
Bool: 8 (old), 8 (new), 0 (No change)
Color: 767 (old), 767 (new), 0 (No change)
Dimen: 722 (old), 722 (new), 0 (No change)
Drawable: 333 (old), 333 (new), 0 (No change)
Id: 1148 (old), 1148 (new), 0 (No change)
Integer: 32 (old), 32 (new), 0 (No change)
Interpolator: 11 (old), 11 (new), 0 (No change)
Layout: 327 (old), 327 (new), 0 (No change)
Menu: 1 (old), 1 (new), 0 (No change)
Mipmap: 1 (old), 1 (new), 0 (No change)
Plurals: 10 (old), 10 (new), 0 (No change)
String: 736 (old), 736 (new), 0 (No change)
Style: 457 (old), 457 (new), 0 (No change)
Xml: 1 (old), 1 (new), 0 (No change)

Lesson assets: 111 (old), 111 (new), 0 (No change)

AAB differences

Expand to see AAB specifics

Supported configurations:

hdpi (same)
ldpi (same)
mdpi (same)
tvdpi (same)
xhdpi (same)
xxhdpi (same)
xxxhdpi (same)

Base APK

APK file size: 9 MiB (old), 9 MiB (new), 0 bytes (No change)
APK download size (estimated): 9064 KiB (old), 9064 KiB (new), 16 bytes (Removed)

Configuration hdpi

APK file size: 52 KiB (old), 52 KiB (new), 0 bytes (No change)
APK download size (estimated): 22 KiB (old), 22 KiB (new), 0 bytes (No change)

Configuration ldpi

APK file size: 52 KiB (old), 52 KiB (new), 0 bytes (No change)
APK download size (estimated): 17 KiB (old), 17 KiB (new), 0 bytes (No change)

Configuration mdpi

APK file size: 46 KiB (old), 46 KiB (new), 0 bytes (No change)
APK download size (estimated): 17 KiB (old), 17 KiB (new), 0 bytes (No change)

Configuration tvdpi

APK file size: 90 KiB (old), 90 KiB (new), 0 bytes (No change)
APK download size (estimated): 37 KiB (old), 37 KiB (new), 0 bytes (No change)

Configuration xhdpi

APK file size: 60 KiB (old), 60 KiB (new), 0 bytes (No change)
APK download size (estimated): 27 KiB (old), 27 KiB (new), 0 bytes (No change)

Configuration xxhdpi

APK file size: 69 KiB (old), 69 KiB (new), 0 bytes (No change)
APK download size (estimated): 37 KiB (old), 37 KiB (new), 0 bytes (No change)

Configuration xxxhdpi

APK file size: 71 KiB (old), 71 KiB (new), 0 bytes (No change)
APK download size (estimated): 38 KiB (old), 38 KiB (new), 0 bytes (No change)

Beta

Expand to see flavor specifics

Universal APK

APK file size: 10 MiB (old), 10 MiB (new), 0 bytes (No change)

APK download size (estimated): 9169 KiB (old), 9169 KiB (new), 8 bytes (Added)

Method count: 101341 (old), 101341 (new), 0 (No change)

Features: 2 (old), 2 (new), 0 (No change)

Permissions: 6 (old), 6 (new), 0 (No change)

Resources: 5504 (old), 5504 (new), 0 (No change)

Anim: 39 (old), 39 (new), 0 (No change)
Animator: 18 (old), 18 (new), 0 (No change)
Array: 14 (old), 14 (new), 0 (No change)
Attr: 879 (old), 879 (new), 0 (No change)
Bool: 8 (old), 8 (new), 0 (No change)
Color: 767 (old), 767 (new), 0 (No change)
Dimen: 722 (old), 722 (new), 0 (No change)
Drawable: 333 (old), 333 (new), 0 (No change)
Id: 1148 (old), 1148 (new), 0 (No change)
Integer: 32 (old), 32 (new), 0 (No change)
Interpolator: 11 (old), 11 (new), 0 (No change)
Layout: 327 (old), 327 (new), 0 (No change)
Menu: 1 (old), 1 (new), 0 (No change)
Mipmap: 1 (old), 1 (new), 0 (No change)
Plurals: 10 (old), 10 (new), 0 (No change)
String: 736 (old), 736 (new), 0 (No change)
Style: 457 (old), 457 (new), 0 (No change)
Xml: 1 (old), 1 (new), 0 (No change)

Lesson assets: 111 (old), 111 (new), 0 (No change)

AAB differences

Expand to see AAB specifics

Supported configurations:

hdpi (same)
ldpi (same)
mdpi (same)
tvdpi (same)
xhdpi (same)
xxhdpi (same)
xxxhdpi (same)

Base APK

APK file size: 9 MiB (old), 9 MiB (new), 0 bytes (No change)
APK download size (estimated): 9053 KiB (old), 9053 KiB (new), 4 bytes (Added)

Configuration hdpi

APK file size: 52 KiB (old), 52 KiB (new), 0 bytes (No change)
APK download size (estimated): 22 KiB (old), 22 KiB (new), 0 bytes (No change)

Configuration ldpi

APK file size: 52 KiB (old), 52 KiB (new), 0 bytes (No change)
APK download size (estimated): 17 KiB (old), 17 KiB (new), 0 bytes (No change)

Configuration mdpi

APK file size: 46 KiB (old), 46 KiB (new), 0 bytes (No change)
APK download size (estimated): 17 KiB (old), 17 KiB (new), 0 bytes (No change)

Configuration tvdpi

APK file size: 90 KiB (old), 90 KiB (new), 0 bytes (No change)
APK download size (estimated): 37 KiB (old), 37 KiB (new), 0 bytes (No change)

Configuration xhdpi

APK file size: 60 KiB (old), 60 KiB (new), 0 bytes (No change)
APK download size (estimated): 27 KiB (old), 27 KiB (new), 0 bytes (No change)

Configuration xxhdpi

APK file size: 69 KiB (old), 69 KiB (new), 0 bytes (No change)
APK download size (estimated): 37 KiB (old), 37 KiB (new), 0 bytes (No change)

Configuration xxxhdpi

APK file size: 71 KiB (old), 71 KiB (new), 0 bytes (No change)
APK download size (estimated): 38 KiB (old), 38 KiB (new), 0 bytes (No change)

Ga

Expand to see flavor specifics

Universal APK

APK file size: 10 MiB (old), 10 MiB (new), 0 bytes (No change)

APK download size (estimated): 9169 KiB (old), 9169 KiB (new), 0 bytes (No change)

Method count: 101341 (old), 101341 (new), 0 (No change)

Features: 2 (old), 2 (new), 0 (No change)

Permissions: 6 (old), 6 (new), 0 (No change)

Resources: 5504 (old), 5504 (new), 0 (No change)

Anim: 39 (old), 39 (new), 0 (No change)
Animator: 18 (old), 18 (new), 0 (No change)
Array: 14 (old), 14 (new), 0 (No change)
Attr: 879 (old), 879 (new), 0 (No change)
Bool: 8 (old), 8 (new), 0 (No change)
Color: 767 (old), 767 (new), 0 (No change)
Dimen: 722 (old), 722 (new), 0 (No change)
Drawable: 333 (old), 333 (new), 0 (No change)
Id: 1148 (old), 1148 (new), 0 (No change)
Integer: 32 (old), 32 (new), 0 (No change)
Interpolator: 11 (old), 11 (new), 0 (No change)
Layout: 327 (old), 327 (new), 0 (No change)
Menu: 1 (old), 1 (new), 0 (No change)
Mipmap: 1 (old), 1 (new), 0 (No change)
Plurals: 10 (old), 10 (new), 0 (No change)
String: 736 (old), 736 (new), 0 (No change)
Style: 457 (old), 457 (new), 0 (No change)
Xml: 1 (old), 1 (new), 0 (No change)

Lesson assets: 111 (old), 111 (new), 0 (No change)

AAB differences

Expand to see AAB specifics

Supported configurations:

hdpi (same)
ldpi (same)
mdpi (same)
tvdpi (same)
xhdpi (same)
xxhdpi (same)
xxxhdpi (same)

Base APK

APK file size: 9 MiB (old), 9 MiB (new), 4 bytes (Removed)
APK download size (estimated): 9053 KiB (old), 9053 KiB (new), 9 bytes (Removed)

Configuration hdpi

APK file size: 52 KiB (old), 52 KiB (new), 0 bytes (No change)
APK download size (estimated): 22 KiB (old), 22 KiB (new), 0 bytes (No change)

Configuration ldpi

APK file size: 52 KiB (old), 52 KiB (new), 0 bytes (No change)
APK download size (estimated): 17 KiB (old), 17 KiB (new), 0 bytes (No change)

Configuration mdpi

APK file size: 46 KiB (old), 46 KiB (new), 0 bytes (No change)
APK download size (estimated): 17 KiB (old), 17 KiB (new), 0 bytes (No change)

Configuration tvdpi

APK file size: 90 KiB (old), 90 KiB (new), 0 bytes (No change)
APK download size (estimated): 37 KiB (old), 37 KiB (new), 0 bytes (No change)

Configuration xhdpi

APK file size: 60 KiB (old), 60 KiB (new), 0 bytes (No change)
APK download size (estimated): 27 KiB (old), 27 KiB (new), 0 bytes (No change)

Configuration xxhdpi

APK file size: 69 KiB (old), 69 KiB (new), 0 bytes (No change)
APK download size (estimated): 37 KiB (old), 37 KiB (new), 0 bytes (No change)

Configuration xxxhdpi

APK file size: 71 KiB (old), 71 KiB (new), 0 bytes (No change)
APK download size (estimated): 38 KiB (old), 38 KiB (new), 0 bytes (No change)

Rd4dev · 2024-06-28T03:26:09Z

Thanks for the review @adhiamboperes, addressed all the review comments and updated tests accordingly. Can you PTAL!

oppiabot · 2024-06-28T03:29:13Z

Unassigning @Rd4dev since a re-review was requested. @Rd4dev, please make sure you have addressed all review comments. Thanks!

…ged sha value

adhiamboperes

Thanks @Rd4dev, LGTM!

adhiamboperes · 2024-06-28T08:11:54Z

@seanlip, could you PTAL? I have verified that @BenHenning's changes have been addressed.

oppiabot · 2024-06-28T08:13:17Z

Assigning @BenHenning for code owner reviews. Thanks!

seanlip

Approving for codeowners since Ben is away

Dismissing per Adhiambo's confirmation, since Ben is away

Rd4dev added 30 commits June 13, 2024 20:00

Fix lint new line and kdoc checks

6ec35a7

Execute Bazel Command to run coverage on the test target

1ec224a

Parse the coverage execution result to acquire the generated coverage…

ee92168

… data file path

Implemented functionality to TestBazelWorkspace utility to add source…

075c77c

… file content and its related test content in the provided subpackage

Sample Test data and Listing all contents

d0519d6

Added Test for BazelClient - for sample target and non target executions

b15b7bb

Added Coverage Runner functionality tests

f441d11

Added tests for the RunCoverageForTestTarget utility

cc86309

Replaced Java Test Sources with Kotlin Sources

4a6073b

Fix Static Checks / Lint warnings

13534a1

Fix Buildifier Lint checks

fc52d82

Added tests for TestBazelWorkspaceTest to check addition of source an…

0226baa

…d test files with corresponding build configurations and content

Fix Lint warnings with TestBazelWorkspaceTest

2fe7485

Fix Regex Pattern checks

bd683b5

Prototyped parsing and reading the extracted file data as byte array

d410d17

Updated the runCoverageForTestTarget to parse the data file and read …

ece147a

…the contents as byte array

Updated Bazel Client Tests with updated execution to return byte array

b101ec8

Updated BazelClientTest for ParseCoverageDataFile and readDatFileAsBi…

5db0650

…nary

Updated CoverageRunnerTest to check with updated getCoverage result r…

87be114

…eturning coverage data

Updated RunCoverageForTestTargetTest to check the returned coverage data

456e81b

Update kdoc to match the function paramaters

2c65524

Removing suspend while keeping the async operations

62f9e1d

Updated RunCoverageForTestTarget to take in custom process timeout as…

5e6d2ef

… arguments and use them

Initialized bazelClient using lazy field

31b6401

Adjusted visibility and removed unnecessary documentation for private…

81d6855

… methods

Revoking visibility of bazelClient functions as that is called outsid…

6cb1da7

…e in testing the utilities

Fix Lint Check issues

b41fb01

Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…

e66ac12

…o code_coverage_bazel_command_execution

Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…

c646840

…o code_coverage_bazel_command_execution

Introduce new script RunCoverage.kt to later take in source filename …

1ffc83d

…instead of test target

Rd4dev added 3 commits June 27, 2024 10:24

Fixed failing tests and fixed kdoc checks

821725f

Fix test cases that failed due to wrong output path

2b97b51

Fix test case that had mis-matched outputs

d6fb599

Base automatically changed from code_coverage_store_as_proto to develop June 27, 2024 08:31

Rd4dev added 2 commits June 27, 2024 14:15

Resolved conflicts with develop branch

036ab34

Fix Lint checks on unused imports

032abcd

adhiamboperes requested changes Jun 27, 2024

View reviewed changes

oppiabot bot unassigned adhiamboperes Jun 27, 2024

adhiamboperes reviewed Jun 27, 2024

View reviewed changes

scripts/src/java/org/oppia/android/scripts/coverage/RunCoverage.kt Outdated Show resolved Hide resolved

scripts/src/java/org/oppia/android/scripts/coverage/CoverageReporter.kt Outdated Show resolved Hide resolved

Rd4dev added 2 commits June 28, 2024 08:25

Refactor to ignore case formats and updated the html styling with tests

f080cba

Updated RunCoverageTest with update to html stylings

a2273ec

oppiabot bot assigned adhiamboperes and unassigned Rd4dev Jun 28, 2024

Fixed failing test case that failed due to changes in file and unchan…

5d0bb7e

…ged sha value

adhiamboperes approved these changes Jun 28, 2024

View reviewed changes

adhiamboperes assigned seanlip and unassigned adhiamboperes Jun 28, 2024

oppiabot bot assigned BenHenning Jun 28, 2024

seanlip approved these changes Jun 28, 2024

View reviewed changes

seanlip merged commit f53c7e5 into develop Jun 28, 2024
25 checks passed

seanlip deleted the code_coverage_generate_report branch June 28, 2024 15:46

Rd4dev mentioned this pull request Jul 25, 2024

Fix part of #5343: Enable Coverage Generation for a list of files #5461

Merged

6 tasks

Fix part of #5343: Generate Code Coverage Report in HTML and MARKDOWN formats #5443

Fix part of #5343: Generate Code Coverage Report in HTML and MARKDOWN formats #5443

Conversation

Rd4dev commented Jun 24, 2024 • edited Loading

Explanation

Project

Changes Made

Essential Checklist

For UI-specific PRs only

Rd4dev commented Jun 27, 2024

adhiamboperes left a comment

Choose a reason for hiding this comment

oppiabot bot commented Jun 27, 2024

adhiamboperes left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 28, 2024

APK & AAB differences analysis

Dev

Universal APK

AAB differences

Base APK

Configuration hdpi

Configuration ldpi

Configuration mdpi

Configuration tvdpi

Configuration xhdpi

Configuration xxhdpi

Configuration xxxhdpi

Alpha

Universal APK

AAB differences

Base APK

Configuration hdpi

Configuration ldpi

Configuration mdpi

Configuration tvdpi

Configuration xhdpi

Configuration xxhdpi

Configuration xxxhdpi

Beta

Universal APK

AAB differences

Base APK

Configuration hdpi

Configuration ldpi

Configuration mdpi

Configuration tvdpi

Configuration xhdpi

Configuration xxhdpi

Configuration xxxhdpi

Ga

Universal APK

AAB differences

Base APK

Configuration hdpi

Configuration ldpi

Configuration mdpi

Configuration tvdpi

Configuration xhdpi

Configuration xxhdpi

Configuration xxxhdpi

Rd4dev commented Jun 28, 2024

oppiabot bot commented Jun 28, 2024

adhiamboperes left a comment

Choose a reason for hiding this comment

adhiamboperes commented Jun 28, 2024

oppiabot bot commented Jun 28, 2024

seanlip left a comment

Choose a reason for hiding this comment

Rd4dev commented Jun 24, 2024 •

edited

Loading