From add9f7475857dd428884c861aef5c7c9db58f890 Mon Sep 17 00:00:00 2001 From: Ben Henning Date: Wed, 12 Jun 2024 23:08:57 -0700 Subject: [PATCH] Fix #5370, part of #59: Migrate to Bazel 6.5.0 (#4886) ## Explanation Fixes #5370 Fixes part of #59 This PR updates the project to use Bazel 6.5.0 instead of 4.0.0. Note that most of the changes done so far in addressing #59 are centered around the concept of simplifying the Bazel maintenance as much as possible so that it's not too much more difficult than Gradle by the time we fully remove Gradle support from the project. While Bazel will always require more effort, there are many things that can be done to narrow the gap. This is a major step in that process since Bazel 4.x required using a custom Android toolchain (https://github.com/oppia/oppia-bazel-tools) which is not at all user friendly. Plus, there are many compatibility and performance improvements in later versions of Bazel that we want to be able to incorporate within the broader Oppia Android project. Bazel 6.x was specifically chosen because: - Bazel 4.x was missing support for the new D8 version which made it impossible to upgrade past 29.0.2 build tools https://github.com/bazelbuild/bazel/issues/13989. - Bazel 5.x had some additional compatibility issues with the D8 change, so we weren't able to use it, either: https://github.com/bazelbuild/bazel/issues/15957. - Bazel 7.x (which wasn't released when this work was originally done) introduces new bzlmod support that causes some additional build headaches that can be figured out later. - Bazel 6.5.0 specifically was chosen since it's the latest 6.x version (as of this edit) and seems to work correctly with existing unit tests. Some other important details to note: - rules_kotlin 1.7.x is needed at a minimum for Bazel 5.x+ support. However, an additional fix was needed (https://github.com/bazelbuild/rules_kotlin/pull/940) in order to fix a deviation in functionality that occurred starting in Bazel 5.x's java_plugin support which led to some file duplication in rules_kotlin (that was fortunately easy to fix). Unfortunately, this change wasn't backported to 1.7.x so this PR makes use of a custom patch to rules_kotlin 1.7.1 (https://github.com/oppia/rules_kotlin) that includes the needed change. We'll get this change properly once we can upgrade to 1.8.x, though that will also require updating Kotlin itself to 1.8.x due to https://github.com/bazelbuild/rules_kotlin/issues/1019. - Bazel 6.x (maybe 5.x) requires at least build tools 30.0.0 since it completely removed support for the old D8 compat dexer. 32.0.0 was chosen in this PR as it's simply a newer, more up-to-date build tools (and removes D8 completely). With this upgrade to Bazel 6.x we'll be able to update the build tools version more often (so long as it doesn't introduce AGP incompatibilities since we can't upgrade Gradle). - As of Bazel 6.x, we're able to reenable Java header compilation and incremental dexing, both of which should have _significant_ performance improvements for incremental builds of the app (and in fact we will have build errors if we disable incremental dexing). - In CI, we opted to **not** support build tools 29.0.2 or old builds of the app. Instead, we'll rely on build tools failing for certain PRs as an indicator that those PRs will require an update (once this PR is merged) in order to have CI run correctly. This is a lot easier than trying to figure out how to support before/after changes with some fairly complex environment differences. - There are a bunch of version updates that were needed to support the minimum version of Kotlin for rules 1.7.x (1.6 I think) as well as JDK 11 (which I think was needed for Bazel 5.x), and these have largely been taken care of in previous PRs to this one (though the JDK 11 update in CI was done in this PR, along with wiki documentation updates to address #5370). One such case of a necessary version upgrade: https://github.com/google/dagger/issues/2511. - There was a change needed for the databinding java_plugin declaration to specify that it generates an API (in order for it to be used correctly in builds). - rules_java needed to be updated to support the newer version of Bazel. - The desugaring hack needed for kotlinx-coroutines-core-jvm was removed since it's no longer needed with the build tools & Bazel upgrade introduced in this PR. - This includes one small change in third-party to change all single-export wrappers that don't have additional plugins being enabled to aliases instead. This is more semantically correct as the wrappers may lose information (which caused problems when investigating adding Jetpack Compose support in #5401). While this isn't directly required for the Bazel upgrade, this is the last PR needed for Jetpack Compose support so it's being added here for simplicity. - ``.bazelrc`` was updated to configure tools, tests, and builds to all use the remote JDK 11 available via Bazel rather than ever using the user's local JDK. This should improve build hermeticity and consistency across different user environments (see https://bazel.build/docs/bazel-and-java). - Setup docs were updated to remove setting up JDK 11 (or Java at all for Linux & Mac) now that the user no longer needs to install Java (see previous point) except for Windows. The Python instructions were also removed since Bazel 6.x includes fixes for Android tools that previously depended on Python 2.x. - CI was unchanged for Java setup since, as far as I can tell, it's still needed for sdkmanager. There was also some small cleanup in unit_tests.yml that I noticed when updating CI versions. ## Essential Checklist - [x] The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".) - [x] Any changes to [scripts/assets](https://github.com/oppia/oppia-android/tree/develop/scripts/assets) files have their rationale included in the PR explanation. - [x] The PR follows the [style guide](https://github.com/oppia/oppia-android/wiki/Coding-style-guide). - [x] The PR does not contain any unnecessary code changes from Android Studio ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#undo-unnecessary-changes)). - [x] The PR is made from a branch that's **not** called "develop" and is up-to-date with "develop". - [x] The PR is **assigned** to the appropriate reviewers ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#clarification-regarding-assignees-and-reviewers-section)). ## For UI-specific PRs only N/A -- This is a build infrastructure change. It shouldn't impact the end user experience. --------- Co-authored-by: Adhiambo Peres <59600948+adhiamboperes@users.noreply.github.com> Co-authored-by: Sean Lip --- .bazelrc | 9 +- .bazelversion | 2 +- .../action.yml | 29 ++---- .github/workflows/build_tests.yml | 30 +++--- .github/workflows/issue_checks.yml | 2 +- .github/workflows/static_checks.yml | 9 +- .github/workflows/stats.yml | 6 +- .github/workflows/unit_tests.yml | 18 ++-- WORKSPACE | 17 +--- build_vars.bzl | 2 +- config/kitkat_main_dex_class_list.txt | 1 - .../file_content_validation_checks.textproto | 2 +- scripts/assets/maven_dependencies.textproto | 11 +++ .../scripts/testing/TestBazelWorkspace.kt | 2 +- .../common/AndroidBuildSdkPropertiesTest.kt | 2 +- .../scripts/testing/TestBazelWorkspaceTest.kt | 10 +- third_party/BUILD.bazel | 92 ++++--------------- third_party/maven_install.json | 2 +- third_party/versions.bzl | 13 +-- tools/android/BUILD.bazel | 1 + tools/kotlin/BUILD.bazel | 8 +- tools/kotlin/add_kotlinc_optin_support.patch | 59 ------------ .../kotlin/remove_processor_duplicates.patch | 17 ++++ wiki/Bazel-Setup-Instructions-for-Linux.md | 58 ++++-------- wiki/Bazel-Setup-Instructions-for-Mac.md | 65 ++++--------- wiki/Bazel-Setup-Instructions-for-Windows.md | 83 ++--------------- wiki/Installing-Oppia-Android.md | 18 ++-- wiki/Oppia-Bazel-Setup-Instructions.md | 2 +- wiki/Troubleshooting-Installation.md | 23 +++-- 29 files changed, 176 insertions(+), 417 deletions(-) delete mode 100644 tools/kotlin/add_kotlinc_optin_support.patch create mode 100644 tools/kotlin/remove_processor_duplicates.patch diff --git a/.bazelrc b/.bazelrc index f988dcc364a..5ca45844311 100644 --- a/.bazelrc +++ b/.bazelrc @@ -1,11 +1,12 @@ -# Configurations for arguments that should automatically be added to Bazel commands. +# Configurations to ensure that Android-specific classes build correctly. build --android_databinding_use_v3_4_args \ --experimental_android_databinding_v2 \ - --java_header_compilation=false \ - --noincremental_dexing \ - --define=android_standalone_dexing_tool=d8_compat_dx \ --android_databinding_use_androidx +# Ensure that all builds use the same JDK for building & running (for better hermeticity and fewer +# inconsistencies across environment configurations). +build --java_runtime_version=remotejdk_11 --tool_java_runtime_version=remotejdk_11 + # Ensure all built Java files treat warnings as errors (similar to the Kotlin configuration) to help # reduce code smell & potential bugs during development. build --javacopt="-Werror" diff --git a/.bazelversion b/.bazelversion index fcdb2e109f6..f22d756da39 100644 --- a/.bazelversion +++ b/.bazelversion @@ -1 +1 @@ -4.0.0 +6.5.0 diff --git a/.github/actions/set-up-android-bazel-build-environment/action.yml b/.github/actions/set-up-android-bazel-build-environment/action.yml index 6afe9f398a8..fc24770010a 100644 --- a/.github/actions/set-up-android-bazel-build-environment/action.yml +++ b/.github/actions/set-up-android-bazel-build-environment/action.yml @@ -1,7 +1,7 @@ # Contains common operations to set up a hermetic Android + Bazel build environment for Oppia # Android CI workflows. Action prerequisites: -# - JDK 9 must be installed & set as the default version via JAVA_HOME -# - Bazel must be installed, in the path, and be version 4.0.0 +# - JDK 11 must be installed & set as the default version via JAVA_HOME +# - Bazel must be installed, in the path, and be version 6.5.0 # TODO(#1861): Revert SDK pinning for improved CI performance once Bazel is sufficiently stable that # we can rely on the automatic SDK provided by GitHub's CI environment. @@ -19,20 +19,20 @@ runs: $JAVA_HOME/bin/java -version # Verify that the correct version of Java is installed. - java -version 2>&1 | grep -q -E "1.9|9.0" + java -version 2>&1 | grep -q -E "11.0" HAS_CORRECT_JAVA_VERSION=$(echo $?) if [[ "$HAS_CORRECT_JAVA_VERSION" == 1 ]] ; then - echo "Expected Java 9 to be installed" + echo "Expected Java 11 to be installed" exit 1 fi shell: bash - name: Verify Bazel version run: | - bazel --version | grep -q 4.0.0 + bazel --version | grep -q 6.5.0 HAS_CORRECT_BAZEL_VERSION=$(echo $?) if [[ "$HAS_CORRECT_JAVA_VERSION" == 1 ]] ; then - echo "Expected Bazel 4.0.0 to be installed" + echo "Expected Bazel 6.5.0 to be installed" exit 1 fi shell: bash @@ -77,14 +77,9 @@ runs: $ANDROID_HOME/cmdline-tools/tools/bin/sdkmanager --install "platforms;android-33" shell: bash - - name: Install build tools 29.0.2 + - name: Install build tools 32.0.0 run: | - $ANDROID_HOME/cmdline-tools/tools/bin/sdkmanager --install "build-tools;29.0.2" - shell: bash - - - name: Configure Bazel to use JDK 9 for building - run: | - echo "build --java_language_version=9" >> $HOME/.bazelrc + $ANDROID_HOME/cmdline-tools/tools/bin/sdkmanager --install "build-tools;32.0.0" shell: bash - name: Configure Bazel to use specific sandbox tmpfs @@ -92,11 +87,3 @@ runs: echo "build --enable_platform_specific_config" >> $HOME/.bazelrc echo "build:linux --sandbox_tmpfs_path=/tmp" >> $HOME/.bazelrc shell: bash - - - name: Set up Oppia Bazel Android Tools - run: | - mkdir $HOME/opensource - cd $HOME/opensource - git clone https://github.com/oppia/oppia-bazel-tools - echo build --override_repository=android_tools="$(cd "$(dirname "$HOME/opensource/oppia-bazel-tools")"; pwd)/$(basename "$HOME/opensource/oppia-bazel-tools")" >> $HOME/.bazelrc - shell: bash diff --git a/.github/workflows/build_tests.yml b/.github/workflows/build_tests.yml index f61837598b8..9e92486849b 100644 --- a/.github/workflows/build_tests.yml +++ b/.github/workflows/build_tests.yml @@ -25,15 +25,15 @@ jobs: with: fetch-depth: 0 - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: Set up build environment uses: ./.github/actions/set-up-android-bazel-build-environment @@ -168,15 +168,15 @@ jobs: with: fetch-depth: 0 - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: Set up build environment uses: ./.github/actions/set-up-android-bazel-build-environment @@ -302,15 +302,15 @@ jobs: with: fetch-depth: 0 - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: Set up build environment uses: ./.github/actions/set-up-android-bazel-build-environment @@ -449,15 +449,15 @@ jobs: with: fetch-depth: 0 - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: Set up build environment uses: ./.github/actions/set-up-android-bazel-build-environment @@ -570,15 +570,15 @@ jobs: with: fetch-depth: 0 - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: Set up build environment uses: ./.github/actions/set-up-android-bazel-build-environment diff --git a/.github/workflows/issue_checks.yml b/.github/workflows/issue_checks.yml index 65d2daa2570..a810002a8e2 100644 --- a/.github/workflows/issue_checks.yml +++ b/.github/workflows/issue_checks.yml @@ -19,7 +19,7 @@ jobs: - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: TODO Issue Resolved Check id: todoIssueResolvedCheck diff --git a/.github/workflows/static_checks.yml b/.github/workflows/static_checks.yml index cd99def2b9d..b312a43f067 100644 --- a/.github/workflows/static_checks.yml +++ b/.github/workflows/static_checks.yml @@ -51,11 +51,10 @@ jobs: - name: Create oppia android tools directory run: mkdir -p $HOME/oppia-android-tools - # Java 11 is specifically needed for Checkstyle. - - name: Set up JDK 1.11 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 1.11 + java-version: 11 - name: Download Checkstyle run: | @@ -104,7 +103,7 @@ jobs: - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - uses: actions/cache@v2 id: scripts_cache @@ -196,7 +195,7 @@ jobs: - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - name: Maven Repin Check if: always() diff --git a/.github/workflows/stats.yml b/.github/workflows/stats.yml index 985d8a2ac5d..de3ce5fadd5 100644 --- a/.github/workflows/stats.yml +++ b/.github/workflows/stats.yml @@ -56,15 +56,15 @@ jobs: run: | echo "PR $PR_NUMBER is merging into $PR_BASE_REF_NAME from https://github.com/$PR_HEAD branch $PR_HEAD_REF_NAME." - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 # For reference on this & the later cache actions, see: # https://github.com/actions/cache/issues/239#issuecomment-606950711 & diff --git a/.github/workflows/unit_tests.yml b/.github/workflows/unit_tests.yml index 5256513d790..fb8436dcc96 100644 --- a/.github/workflows/unit_tests.yml +++ b/.github/workflows/unit_tests.yml @@ -18,7 +18,7 @@ jobs: runs-on: ubuntu-20.04 outputs: matrix: ${{ steps.compute-test-matrix.outputs.matrix }} - have_tests_to_run: ${{ steps.compute-test-matrix.outputs.have_tests_to_run }} + can_skip_tests: ${{ steps.compute-test-matrix.outputs.can_skip_tests }} env: CACHE_DIRECTORY: ~/.bazel_cache steps: @@ -29,7 +29,7 @@ jobs: - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - uses: actions/cache@v2 id: scripts_cache @@ -86,16 +86,16 @@ jobs: echo "Affected tests (note that this might be all tests if configured to run all or on the develop branch): $TEST_BUCKET_LIST" echo "::set-output name=matrix::{\"affected-tests-bucket-base64-encoded-shard\":[$TEST_BUCKET_LIST]}" if [[ ! -z "$TEST_BUCKET_LIST" ]]; then - echo "::set-output name=have_tests_to_run::true" + echo "::set-output name=can_skip_tests::false" else - echo "::set-output name=have_tests_to_run::false" + echo "::set-output name=can_skip_tests::true" echo "No tests are detected as affected by this change. If this is wrong, you can add '[RunAllTests]' to the PR title to force a run." fi bazel_run_test: name: Run Bazel Test needs: bazel_compute_affected_targets - if: ${{ needs.bazel_compute_affected_targets.outputs.have_tests_to_run == 'true' }} + if: ${{ needs.bazel_compute_affected_targets.outputs.can_skip_tests != 'true' }} runs-on: ubuntu-20.04 strategy: fail-fast: false @@ -107,15 +107,15 @@ jobs: steps: - uses: actions/checkout@v2 - - name: Set up JDK 9 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: 9 + java-version: 11 - name: Set up Bazel uses: abhinavsingh/setup-bazel@v3 with: - version: 4.0.0 + version: 6.5.0 - uses: actions/cache@v2 id: scripts_cache @@ -325,5 +325,5 @@ jobs: steps: # This step will be skipped if there are no tests to run, so the overall job should pass. - name: Check tests passed (for tests that ran) - if: ${{ needs.bazel_compute_affected_targets.outputs.have_tests_to_run == 'true' && needs.bazel_run_test.result != 'success' }} + if: ${{ needs.bazel_compute_affected_targets.outputs.can_skip_tests != 'true' && needs.bazel_run_test.result != 'success' }} run: exit 1 diff --git a/WORKSPACE b/WORKSPACE index 1232f4971ea..e877ad78248 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -43,14 +43,14 @@ http_archive( # Add support for Kotlin: https://github.com/bazelbuild/rules_kotlin. http_archive( name = "io_bazel_rules_kotlin", - patches = ["//tools/kotlin:add_kotlinc_optin_support.patch"], + patches = ["//tools/kotlin:remove_processor_duplicates.patch"], sha256 = HTTP_DEPENDENCY_VERSIONS["rules_kotlin"]["sha"], urls = ["https://github.com/bazelbuild/rules_kotlin/releases/download/%s/rules_kotlin_release.tgz" % HTTP_DEPENDENCY_VERSIONS["rules_kotlin"]["version"]], ) load("@io_bazel_rules_kotlin//kotlin:repositories.bzl", "kotlin_repositories", "kotlinc_version") -# Use the 1.6 compiler since rules_kotlin 1.5 defaults to the 1.5 compiler. +# Use the 1.6 compiler since Kotlin 1.6 is the current supported version in the repository. kotlin_repositories( compiler_release = kotlinc_version( release = "1.6.10", @@ -212,7 +212,6 @@ maven_install( maven_install_json = "//third_party:maven_install.json", override_targets = { "com.google.guava:guava": "@//third_party:com_google_guava_guava", - "org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm": "@//third_party:kotlinx-coroutines-core-jvm", }, repositories = DAGGER_REPOSITORIES + MAVEN_REPOSITORIES, strict_visibility = True, @@ -240,15 +239,3 @@ pinned_maven_install() "jre", ] ] - -http_jar( - name = "kotlinx-coroutines-core-jvm", - sha256 = HTTP_DEPENDENCY_VERSIONS["kotlinx-coroutines-core-jvm"]["sha"], - urls = [ - "{0}/org/jetbrains/kotlinx/kotlinx-coroutines-core-jvm/{1}/kotlinx-coroutines-core-jvm-{1}.jar".format( - url_base, - HTTP_DEPENDENCY_VERSIONS["kotlinx-coroutines-core-jvm"]["version"], - ) - for url_base in DAGGER_REPOSITORIES + MAVEN_REPOSITORIES - ], -) diff --git a/build_vars.bzl b/build_vars.bzl index 99b12eef5ae..6fd23a4f9a5 100644 --- a/build_vars.bzl +++ b/build_vars.bzl @@ -1,2 +1,2 @@ BUILD_SDK_VERSION = 33 -BUILD_TOOLS_VERSION = "29.0.2" +BUILD_TOOLS_VERSION = "32.0.0" diff --git a/config/kitkat_main_dex_class_list.txt b/config/kitkat_main_dex_class_list.txt index a399ae49c61..ea2de317d21 100644 --- a/config/kitkat_main_dex_class_list.txt +++ b/config/kitkat_main_dex_class_list.txt @@ -10,7 +10,6 @@ androidx/multidex/ZipUtil.class androidx/work/Configuration$Provider.class androidx/work/Configuration.class androidx/work/WorkManager.class -com/google/firebase/FirebaseApp.class javax/inject/Provider.class kotlin/Function.class kotlin/jvm/functions/Function0.class diff --git a/scripts/assets/file_content_validation_checks.textproto b/scripts/assets/file_content_validation_checks.textproto index eca06c82432..3c4cebcfe47 100644 --- a/scripts/assets/file_content_validation_checks.textproto +++ b/scripts/assets/file_content_validation_checks.textproto @@ -133,7 +133,7 @@ file_content_checks { } file_content_checks { file_path_regex: ".+?\\.kt" - prohibited_content_regex: "(format|getString|getStringArray|getQuantityString|getQuantityText|toLowerCase|toUpperCase|capitalize|decapitalize|lowercase|uppercase)\\(" + prohibited_content_regex: "(format|getString|getStringArray|getQuantityString|getQuantityText|toLowerCase|toUpperCase|capitalize|decapitalize|lowercase|uppercase|replaceFirstChar)\\(" failure_message: "String formatting and resource retrieval should go through AppLanguageResourceHandler, OppiaLocale.DisplayLocale, or OppiaLocale.MachineLocale depending on the context (see each class's documentation for details on when each should be used)." exempted_file_name: "app/src/main/java/org/oppia/android/app/translation/AppLanguageResourceHandler.kt" exempted_file_name: "domain/src/main/java/org/oppia/android/domain/util/JsonExtensions.kt" diff --git a/scripts/assets/maven_dependencies.textproto b/scripts/assets/maven_dependencies.textproto index eb4db32a1fb..4f046941d1e 100644 --- a/scripts/assets/maven_dependencies.textproto +++ b/scripts/assets/maven_dependencies.textproto @@ -1680,6 +1680,17 @@ maven_dependency { } } } +maven_dependency { + artifact_name: "org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.6.4" + artifact_version: "1.6.4" + license { + license_name: "The Apache Software License, Version 2.0" + original_link: "https://www.apache.org/licenses/LICENSE-2.0.txt" + scrapable_link { + url: "https://www.apache.org/licenses/LICENSE-2.0.txt" + } + } +} maven_dependency { artifact_name: "org.jetbrains.kotlinx:kotlinx-coroutines-guava:1.6.4" artifact_version: "1.6.4" diff --git a/scripts/src/java/org/oppia/android/scripts/testing/TestBazelWorkspace.kt b/scripts/src/java/org/oppia/android/scripts/testing/TestBazelWorkspace.kt index c51df7a7f14..7c57cf0d10c 100644 --- a/scripts/src/java/org/oppia/android/scripts/testing/TestBazelWorkspace.kt +++ b/scripts/src/java/org/oppia/android/scripts/testing/TestBazelWorkspace.kt @@ -5,7 +5,7 @@ import org.junit.rules.TemporaryFolder import java.io.File /** The version of Bazel to use in tests that set up Bazel workspaces. */ -const val BAZEL_VERSION = "4.0.0" +const val BAZEL_VERSION = "6.5.0" /** * Test utility for generating various test & library targets in the specified [TemporaryFolder]. diff --git a/scripts/src/javatests/org/oppia/android/scripts/common/AndroidBuildSdkPropertiesTest.kt b/scripts/src/javatests/org/oppia/android/scripts/common/AndroidBuildSdkPropertiesTest.kt index d7cb9a5ebe9..be621bd982c 100644 --- a/scripts/src/javatests/org/oppia/android/scripts/common/AndroidBuildSdkPropertiesTest.kt +++ b/scripts/src/javatests/org/oppia/android/scripts/common/AndroidBuildSdkPropertiesTest.kt @@ -26,6 +26,6 @@ class AndroidBuildSdkPropertiesTest { fun testBuildToolsVersion_isTheCorrectVersion() { val properties = AndroidBuildSdkProperties() - assertThat(properties.buildToolsVersion).isEqualTo("29.0.2") + assertThat(properties.buildToolsVersion).isEqualTo("32.0.0") } } diff --git a/scripts/src/javatests/org/oppia/android/scripts/testing/TestBazelWorkspaceTest.kt b/scripts/src/javatests/org/oppia/android/scripts/testing/TestBazelWorkspaceTest.kt index cab5a24d7ef..2ea2f42b1bf 100644 --- a/scripts/src/javatests/org/oppia/android/scripts/testing/TestBazelWorkspaceTest.kt +++ b/scripts/src/javatests/org/oppia/android/scripts/testing/TestBazelWorkspaceTest.kt @@ -55,7 +55,7 @@ class TestBazelWorkspaceTest { // A .bazelversion file should now exist with the correct version. val bazelVersionFile = File(tempFolder.root, ".bazelversion") assertThat(bazelVersionFile.exists()).isTrue() - assertThat(bazelVersionFile.readText().trim()).isEqualTo("4.0.0") + assertThat(bazelVersionFile.readText().trim()).isEqualTo("6.5.0") } @Test @@ -120,7 +120,7 @@ class TestBazelWorkspaceTest { ) val bazelVersionContent = tempFolder.getBazelVersionFile().readText().trim() - assertThat(bazelVersionContent).isEqualTo("4.0.0") + assertThat(bazelVersionContent).isEqualTo("6.5.0") } @Test @@ -292,7 +292,7 @@ class TestBazelWorkspaceTest { ) val bazelVersionContent = tempFolder.getBazelVersionFile().readText().trim() - assertThat(bazelVersionContent).isEqualTo("4.0.0") + assertThat(bazelVersionContent).isEqualTo("6.5.0") } @Test @@ -570,7 +570,7 @@ class TestBazelWorkspaceTest { testBazelWorkspace.createTest(testName = "FirstTest") val bazelVersionContent = tempFolder.getBazelVersionFile().readText().trim() - assertThat(bazelVersionContent).isEqualTo("4.0.0") + assertThat(bazelVersionContent).isEqualTo("6.5.0") } @Test @@ -790,7 +790,7 @@ class TestBazelWorkspaceTest { testBazelWorkspace.createLibrary(dependencyName = "ExampleDep") val bazelVersionContent = tempFolder.getBazelVersionFile().readText().trim() - assertThat(bazelVersionContent).isEqualTo("4.0.0") + assertThat(bazelVersionContent).isEqualTo("6.5.0") } @Test diff --git a/third_party/BUILD.bazel b/third_party/BUILD.bazel index 74f84964b65..5c8dc488a58 100644 --- a/third_party/BUILD.bazel +++ b/third_party/BUILD.bazel @@ -12,7 +12,6 @@ own Bazel macros to automatically set up code generation (which includes pulling dependencies). """ -load("@io_bazel_rules_kotlin//kotlin:jvm.bzl", "kt_jvm_import", "kt_jvm_library") load("@rules_java//java:defs.bzl", "java_binary", "java_library") load("@rules_jvm_external//:defs.bzl", "artifact") load(":versions.bzl", "MAVEN_PRODUCTION_DEPENDENCY_VERSIONS", "MAVEN_TEST_DEPENDENCY_VERSIONS") @@ -20,57 +19,45 @@ load(":versions.bzl", "MAVEN_PRODUCTION_DEPENDENCY_VERSIONS", "MAVEN_TEST_DEPEND # Create an android library wrapper for all production dependencies. Note that artifact is used so # that the correct Maven repository (default) is selected. This allows future customization # potential in managing multiple Maven repositories. -[android_library( +[alias( name = name.replace(":", "_").replace(".", "_"), + actual = artifact("%s:%s" % (name, version)), visibility = ["//visibility:public"], - exports = [ - artifact("%s:%s" % (name, version)), - ], ) for name, version in MAVEN_PRODUCTION_DEPENDENCY_VERSIONS.items()] # Create an android library wrapper for all test dependencies. -[android_library( +[alias( name = name.replace(":", "_").replace(".", "_"), testonly = True, + actual = artifact("%s:%s" % (name, version)), visibility = ["//visibility:public"], - exports = [ - artifact("%s:%s" % (name, version)), - ], ) for name, version in MAVEN_TEST_DEPENDENCY_VERSIONS.items()] # A wrapper dependency for the custom Git import of Android SVG (so that dependencies on this # library still routes through //third_party). -android_library( +alias( name = "com_caverock_androidsvg", + actual = "@androidsvg//androidsvg", visibility = ["//visibility:public"], - exports = [ - "@androidsvg//androidsvg", - ], ) -android_library( +alias( name = "com_github_takusemba_spotlight", + actual = "@android-spotlight//spotlight", visibility = ["//visibility:public"], - exports = [ - "@android-spotlight//spotlight", - ], ) -android_library( +alias( name = "robolectric_android-all", testonly = True, + actual = "@robolectric//bazel:android-all", visibility = ["//visibility:public"], - exports = [ - "@robolectric//bazel:android-all", - ], ) -android_library( +alias( name = "com_google_archivepatcher", + actual = "@archive_patcher//tools", visibility = ["//visibility:public"], - exports = [ - "@archive_patcher//tools", - ], ) java_library( @@ -82,12 +69,10 @@ java_library( ], ) -android_library( +alias( name = "io_github_karino2_kotlitex", + actual = "@kotlitex//kotlitex", visibility = ["//visibility:public"], - exports = [ - "@kotlitex//kotlitex", - ], ) alias( @@ -118,28 +103,6 @@ android_library( ], ) -# Note that the output from kt_jvm_import needs to be wrapped in a kt_jvm_library in order to work -# with the Bazel IntelliJ plugin correctly (since kt_jvm_library provides the expected IDE output -# for syncing). -kt_jvm_import( - name = "_kotlinx-coroutines-core-jvm_import_do_not_depend", - jars = ["@kotlinx-coroutines-core-jvm//jar:file"], - tags = ["no-ide"], - deps = [ - ":_stopgap_lib_for_kotlinx_coroutines_desugar_do_not_depend", - "//third_party:org_jetbrains_kotlin_kotlin-stdlib-jdk8_jar", - ], -) - -kt_jvm_library( - name = "kotlinx-coroutines-core-jvm", - visibility = [ - "//:oppia_api_visibility", - "@maven//:__subpackages__", - ], - exports = [":_kotlinx-coroutines-core-jvm_import_do_not_depend"], -) - java_library( name = "com_google_guava_guava_jre", visibility = ["//scripts:__subpackages__"], @@ -153,29 +116,6 @@ java_library( ], ) -# This solution is based on https://github.com/Bencodes/bazel_issue_13553 for Bazel issue: -# https://github.com/bazelbuild/bazel/issues/13553. -genrule( - name = "_generate_stopgap_files_for_kotlinx_coroutines_desugar", - outs = [ - "sun/misc/Signal.java", - "sun/misc/SignalHandler.java", - ], - cmd = """ - echo "package sun.misc; public final class Signal {}" > $(location sun/misc/Signal.java) - echo "package sun.misc; public interface SignalHandler {}" > $(location sun/misc/SignalHandler.java) - """, -) - -java_library( - name = "_stopgap_lib_for_kotlinx_coroutines_desugar_do_not_depend", - srcs = [ - "sun/misc/Signal.java", - "sun/misc/SignalHandler.java", - ], - neverlink = True, -) - # Define a separate target for the Glide annotation processor compiler. Unfortunately, this library # can't encapsulate all of Glide (i.e. by exporting the main Glide dependency) since that includes # Android assets which java_library targets do not export. @@ -185,10 +125,10 @@ java_library( visibility = ["//visibility:public"], ) -android_library( +alias( name = "android_bundletool", + actual = "@android_bundletool//jar", visibility = ["//visibility:public"], - exports = ["@android_bundletool//jar"], ) java_binary( diff --git a/third_party/maven_install.json b/third_party/maven_install.json index f25b55f9094..c9a27f18450 100644 --- a/third_party/maven_install.json +++ b/third_party/maven_install.json @@ -1,6 +1,6 @@ { "__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": "THERE_IS_NO_DATA_ONLY_ZUUL", - "__INPUT_ARTIFACTS_HASH": 1828687201, + "__INPUT_ARTIFACTS_HASH": 1041130706, "__RESOLVED_ARTIFACTS_HASH": 1257218600, "conflict_resolution": { "androidx.constraintlayout:constraintlayout:1.1.3": "androidx.constraintlayout:constraintlayout:2.0.1", diff --git a/third_party/versions.bzl b/third_party/versions.bzl index a90a2d476c6..f9e0864672d 100644 --- a/third_party/versions.bzl +++ b/third_party/versions.bzl @@ -81,6 +81,7 @@ MAVEN_PRODUCTION_DEPENDENCY_VERSIONS = { "org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar": "1.6.10", "org.jetbrains.kotlinx:kotlinx-coroutines-android": "1.6.4", "org.jetbrains.kotlinx:kotlinx-coroutines-core": "1.6.4", + "org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm": "1.6.4", "org.jetbrains.kotlinx:kotlinx-coroutines-guava": "1.6.4", "org.jetbrains:annotations:jar": "13.0", } @@ -140,10 +141,6 @@ HTTP_DEPENDENCY_VERSIONS = { "sha": "d5be94d65e87bd219fb3193ad1517baa55a3b88fc91d21cf735826ab5af087b9", "version": "31.0.1", }, - "kotlinx-coroutines-core-jvm": { - "sha": "acc8c74b1fb88121c51221bfa7b6f5e920201bc20183ebf74165dcf5d45a8003", - "version": "1.6.0", - }, "oppia_proto_api": { "version": "9cf993ea0b798a67b3faa21c690c30b9027fb371", }, @@ -156,16 +153,16 @@ HTTP_DEPENDENCY_VERSIONS = { "version": "4.5", }, "rules_java": { - "sha": "34b41ec683e67253043ab1a3d1e8b7c61e4e8edefbcad485381328c934d072fe", - "version": "4.0.0", + "sha": "c73336802d0b4882e40770666ad055212df4ea62cfa6edf9cb0f9d29828a0934", + "version": "5.3.5", }, "rules_jvm": { "sha": "c4cd0fd413b43785494b986fdfeec5bb47eddca196af5a2a98061faab83ed7b2", "version": "5.1", }, "rules_kotlin": { - "sha": "12d22a3d9cbcf00f2e2d8f0683ba87d3823cb8c7f6837568dd7e48846e023307", - "version": "v1.5.0", + "sha": "fd92a98bd8a8f0e1cdcb490b93f5acef1f1727ed992571232d33de42395ca9b3", + "version": "v1.7.1", }, "rules_proto": { "sha": "e0cab008a9cdc2400a1d6572167bf9c5afc72e19ee2b862d18581051efab42c9", diff --git a/tools/android/BUILD.bazel b/tools/android/BUILD.bazel index 97dc9525fb7..010bb615193 100644 --- a/tools/android/BUILD.bazel +++ b/tools/android/BUILD.bazel @@ -6,6 +6,7 @@ load("@rules_java//java:defs.bzl", "java_plugin") java_plugin( name = "compiler_annotation_processor", + generates_api = True, processor_class = "android.databinding.annotationprocessor.ProcessDataBinding", visibility = ["//visibility:public"], deps = [ diff --git a/tools/kotlin/BUILD.bazel b/tools/kotlin/BUILD.bazel index 9348f206bd2..ad4704f6609 100644 --- a/tools/kotlin/BUILD.bazel +++ b/tools/kotlin/BUILD.bazel @@ -5,10 +5,9 @@ Configurations for a codebase-wide build toolchain for Kotlin. load("@bazel_skylib//rules:common_settings.bzl", "string_flag") load("@io_bazel_rules_kotlin//kotlin:core.bzl", "define_kt_toolchain", "kt_javac_options", "kt_kotlinc_options") -# This exposes a patch file that retroactively adds opt-in support to kt_kotlinc_options (which was -# added in a later version of rules_kotlin that's not available for the version of Bazel used at the -# time of this introduction). -exports_files(["add_kotlinc_optin_support.patch"]) +# This exposes a patch that fixes an issue with duplicate processor arguments. See: +# github.com/bazelbuild/rules_kotlin/pull/940. +exports_files(["remove_processor_duplicates.patch"]) string_flag( name = "warn_mode", @@ -36,7 +35,6 @@ kt_kotlinc_options( ":errors_as_warnings": "report", }), x_optin = ["kotlin.RequiresOptIn"], - x_use_experimental = False, ) # TODO(#4936): Fix "warnings as errors" for processor-generated Java code. diff --git a/tools/kotlin/add_kotlinc_optin_support.patch b/tools/kotlin/add_kotlinc_optin_support.patch deleted file mode 100644 index b0c379b673f..00000000000 --- a/tools/kotlin/add_kotlinc_optin_support.patch +++ /dev/null @@ -1,59 +0,0 @@ ---- src/main/starlark/core/repositories/configured_rules.bzl -+++ src/main/starlark/core/repositories/configured_rules.bzl -@@ -1,6 +1,56 @@ - def _rules_repository_impl(repository_ctx): - attr = repository_ctx.attr - repository_ctx.extract(attr.archive) -+ repository_ctx.file( -+ "provided.patch", -+ content = """ -+--- kotlin/opts.bzl -++++ kotlin/opts.bzl -+@@ -1,3 +1,6 @@ -++def _map_optin_class_to_flag(values): -++ return ["-opt-in=%s" % v for v in values] -++ -+ _KOPTS = { -+ "warn": struct( -+ args = dict( -+@@ -121,6 +124,16 @@ -+ True: ["-Xmulti-platform"], -+ }, -+ ), -++ # Add support for opt-in values per https://github.com/bazelbuild/rules_kotlin/blob/0b75e942e444ccced07cfec50cbc8d925821ed57/src/rkt_1_6/starlark/kotlin/opts.bzl#L196-L204. -++ "x_optin": struct( -++ args = dict( -++ default = [], -++ doc = "Define APIs to opt-in to.", -++ ), -++ type = attr.string_list, -++ value_to_flag = None, -++ map_value_to_flag = _map_optin_class_to_flag, -++ ), -+ } -+ -+ KotlincOptions = provider( -+@@ -158,10 +171,14 @@ -+ if not kotlinc_options: -+ return "" -+ -++ # Updated to support new map_value_to_flag per https://github.com/bazelbuild/rules_kotlin/blob/0b75e942e444ccced07cfec50cbc8d925821ed57/src/main/starlark/core/options/convert.bzl#L3. -+ flags = [] -+ for n, o in _KOPTS.items(): -+ value = getattr(kotlinc_options, n, None) -+- flag = o.value_to_flag.get(value, None) -++ if o.value_to_flag: -++ flag = o.value_to_flag.get(value, None) -++ else: -++ flag = o.map_value_to_flag(value) -+ if flag: -+ flags.extend(flag) -+ return flags -+ -+ """, -+ executable = False -+ ) -+ repository_ctx.patch("provided.patch") - - rules_repository = repository_rule( - implementation = _rules_repository_impl, diff --git a/tools/kotlin/remove_processor_duplicates.patch b/tools/kotlin/remove_processor_duplicates.patch new file mode 100644 index 00000000000..ec052a8db91 --- /dev/null +++ b/tools/kotlin/remove_processor_duplicates.patch @@ -0,0 +1,17 @@ +--- kotlin/internal/jvm/compile.bzl ++++ kotlin/internal/jvm/compile.bzl +@@ -363,12 +363,14 @@ def _run_kt_builder_action( + annotation_processors, + map_each = _plugin_mappers.kt_plugin_to_processor, + omit_if_empty = True, ++ uniquify = True, + ) + args.add_all( + "--processorpath", + annotation_processors, + map_each = _plugin_mappers.kt_plugin_to_processorpath, + omit_if_empty = True, ++ uniquify = True, + ) + + compiler_plugins = [ diff --git a/wiki/Bazel-Setup-Instructions-for-Linux.md b/wiki/Bazel-Setup-Instructions-for-Linux.md index 37648fda40d..6410ca034b6 100644 --- a/wiki/Bazel-Setup-Instructions-for-Linux.md +++ b/wiki/Bazel-Setup-Instructions-for-Linux.md @@ -1,58 +1,30 @@ ## Instructions **The steps to install Bazel on Linux are:** -1. Install Bazel -2. Install OpenJDK 8 -3. Install Python 2 and make sure it is active in your environment -4. Set up the ANDROID_HOME environment variable -5. Prepare the build environment -6. Verify the Android build +1. Install Bazel +2. Set up the ANDROID_HOME environment variable +3. Verify the Android build ### 1. Install Bazel -Install Bazel from [here](https://docs.bazel.build/versions/master/install.html). Make sure that you follow the instructions for installing a specific version (Oppia Android requires 4.0.0 and won't build on other versions). +Install Bazel from [here](https://docs.bazel.build/versions/master/install.html). Make sure that you follow the instructions for installing a specific version (Oppia Android requires 6.5.0 and won't build on other versions). - Note: if you find any errors related to `cURL`, please set up cURL on your machine. For Linux, you can use `sudo apt install curl`. -### 2. Install OpenJDK 8 - -Oppia Android also requires OpenJDK 8. The Bazel installation instructions above include [sections on installing OpenJDK](https://docs.bazel.build/versions/main/tutorial/java.html#install-the-jdk) on different platforms. - - - You can run the following to install OpenJDK 8: - - ```sh - sudo apt install openjdk-8-jdk - ``` - - You can confirm that this is set up using the command `java -version`, which should result in three lines being printed out with the first one showing "openjdk version "1.8.0_292". - -> **Tip:** Newer linux distributions may not have openjdk-8 anymore. This has been reported for Debian and Ubuntu. If this is your case, **openjdk-11** has been tested as a viable alternative. - -### 3. Install Python 2 - -Ensure that you have Python 2 installed and make sure that it is currently active on your environment. You can do this by using the ``python --version`` command which should show Python 2.X.X. If it doesn’t, click [here](https://linuxconfig.org/install-python-2-on-ubuntu-20-04-focal-fossa-linux) for a resource on how to install and update Linux to use Python 2. - -> **Tip:** Newer linux distributions may not have the python command and instead use python2 and python3. Bazel <5 requires `python` and that [cannot be configured](https://stackoverflow.com/questions/62802126/how-to-stop-bazel-from-relying-on-python2). Creating an alias won't solve the problem, you'll need a symlink. To create one, you can run the following: -> -> `sudo apt install python-is-python2` - -### 4. Set up the ANDROID_HOME environment variable +### 2. Set up the ANDROID_HOME environment variable Ensure that your `ANDROID_HOME` environment variable is set to the location of your Android SDK. To do this, find the path to the installed SDK using Android Studio’s SDK Manager (install SDK 28). Assuming the SDK is installed to default locations, you can use the following commands to set the `ANDROID_HOME` variable:
``` export ANDROID_HOME=$HOME/Android/Sdk/ ``` - - **Make sure you have the system environment variable set up** for ``ANDROID_HOME`` as you might have issues getting properly set up if not. If it isn’t set up (on Linux you can check by using ``echo $ANDROID_HOME`` in a new terminal; it should output the correct path to your Android SDK), on Linux you can move the ``export`` from above to your ``~/.bashrc`` file to make it permanent (you can apply the change immediately using ``source ~/.bashrc``). - -### 5. Prepare the build environment -Follow the instructions in [oppia-bazel-tools](https://github.com/oppia/oppia-bazel-tools). +**Make sure you have the system environment variable set up** for ``ANDROID_HOME`` as you might have issues getting properly set up if not. If it isn’t set up (on Linux you can check by using ``echo $ANDROID_HOME`` in a new terminal; it should output the correct path to your Android SDK), on Linux you can move the ``export`` from above to your ``~/.bashrc`` file to make it permanent (you can apply the change immediately using ``source ~/.bashrc``). -### 6. Verifying the build +### 3. Verifying the build At this point, your system should be able to build Oppia Android. To verify, try building the APK (from your subsystem terminal -- note that this and all other Bazel commands must be run from the root of the ‘oppia-android’ directory otherwise they will fail): -```sh -bazel build //:oppia +``` +bazel build //:oppia_dev ``` (Note that this command may take 10-20 minutes to complete depending on the performance of your machine). @@ -60,13 +32,15 @@ bazel build //:oppia If everything is working, you should see output like the following: ``` -Target //:oppia up-to-date: - bazel-bin/oppia_deploy.jar - bazel-bin/oppia_unsigned/apk - bazel-bin/oppia/apk +Target //:oppia_dev up-to-date: + bazel-bin/oppia_dev.aab INFO: Elapsed time: ... INFO: 1 process... INFO: Build completed successfully, ... ``` -Note also that the ``oppia.apk`` under the ``bazel-bin`` directory of your local copy of Oppia Android should be a fully functioning development version of the app that can be installed using ``adb`` +Note also that the ``oppia_dev.aab`` under the ``bazel-bin`` directory of your local copy of Oppia Android should be a fully functioning development version of the app that can be installed using bundle-tool. However, it's recommended to deploy Oppia to an emulator or connected device using the following Bazel command: + +```sh +bazel run //:install_oppia_dev +``` diff --git a/wiki/Bazel-Setup-Instructions-for-Mac.md b/wiki/Bazel-Setup-Instructions-for-Mac.md index 11fd0116f8b..f7b3b2e7dfd 100644 --- a/wiki/Bazel-Setup-Instructions-for-Mac.md +++ b/wiki/Bazel-Setup-Instructions-for-Mac.md @@ -4,12 +4,9 @@ **The steps to install Bazel on Mac are:** 1. Set up Rosetta Terminal -2. Install Bazel -3. Install OpenJDK 8 -4. Install Python 2 and make sure it is active in your environment -5. Set Bazel, Python 2, ANDROID_HOME paths permanently in your terminal -6. Prepare the build environment -7. Verify that the build is working +2. Install Bazel +3. Set Bazel and ANDROID_HOME paths permanently in your terminal +4. Verify that the build is working ### 1. Set up Rosetta Terminal @@ -21,7 +18,7 @@ ### 2. Install Bazel -1. Install Bazel following the instructions [here](https://docs.bazel.build/versions/4.0.0/install-os-x.html#install-with-installer-mac-os-x). Make sure that you follow the instructions for installing a specific version (Oppia Android requires 4.0.0 and won't build on other versions). +1. Install Bazel following the instructions [here](https://docs.bazel.build/versions/4.0.0/install-os-x.html#install-with-installer-mac-os-x). Make sure that you follow the instructions for installing a specific version (Oppia Android requires 6.5.0 and won't build on other versions). 2. That’s it, now Bazel is installed, and you can verify it by running the command: ``` @@ -29,61 +26,33 @@ ``` - **Expected Output** ``` - bazel 4.0.0 + bazel 6.5.0 ``` -### 3. Install OpenJDK 8 +### 3. Set Bazel and ANDROID_HOME paths permanently in your terminal -Oppia Android also requires OpenJDK 8. -Follow the instructions [here](https://installvirtual.com/install-openjdk-8-on-mac-using-brew-adoptopenjdk/) to install OpenJDK 8. -Note that this requires the installation of brew as a pre-requisite, which can be done by following the instructions [here](https://mac.install.guide/homebrew/index.html). You can then set up your `$JAVA_HOME` environment variable using these [instructions](https://stackoverflow.com/a/75167958/11396524). - - -### 4. Install Python 2 and make sure it is active in your environment - -To install Python 2 in MacOS follow the follows the commands given below. Note that this requires installation of brew as a pre-requisite, which can be done by following the instructions [here](https://mac.install.guide/homebrew/index.html). -``` -brew install pyenv -pyenv install 2.7.18 -pyenv global 2.7.18 -``` - -- To make sure Python 2 is successfully installed and active in your environment, navigate to the **oppia-android** directory and run the following commands: - -``` -export PATH="$(pyenv root)/shims:${PATH}" -python --version -``` - -### 5. Set Bazel, Python 2, ANDROID_HOME paths permanently in your terminal - -- To set the `Bazel`, `Python 2`, `ANDROID_HOME` path permanently in your terminal run these commands: +- To set the `Bazel` and `ANDROID_HOME` path permanently in your terminal run these commands: ``` sudo nano /etc/paths ``` - Enter your password, when prompted. - - Go to the bottom of the file, and enter these paths + - Go to the bottom of the file, and enter these paths ``` /Users/{YourMacUserName}/bin - $(pyenv root)/shims:${PATH} $HOME/Library/Android/sdk ``` - Hit control-x to quit. - Enter “Y” to save the modified buffer. - That’s it! To test it, in a new terminal window, type: `echo $PATH` -**Note: You must set the path for `Bazel`, `Python 2`, `ANDROID_HOME` before running bazel build for oppia-android, otherwise you will get an error.** - -### 6. Prepare the build environment +**Note: You must set the path for `Bazel` and `ANDROID_HOME` before running bazel build for oppia-android, otherwise you will get an error.** -Follow the instructions in [oppia-bazel-tools](https://github.com/oppia/oppia-bazel-tools#readme), in order to prepare your environment to support Oppia Android builds. - -### 7. Verify that the build is working +### 4. Verify that the build is working At this point, your system should be able to build Oppia Android. To verify, try building the APK (from your subsystem terminal -- note that this and all other Bazel commands must be run from the root of the ‘oppia-android’ directory otherwise they will fail): ``` -bazel build //:oppia +bazel build //:oppia_dev ``` (Note that this command may take 10-20 minutes to complete depending on the performance of your machine). @@ -91,13 +60,15 @@ bazel build //:oppia If everything is working, you should see output like the following: ``` -Target //:oppia up-to-date: - bazel-bin/oppia_deploy.jar - bazel-bin/oppia_unsigned/apk - bazel-bin/oppia/apk +Target //:oppia_dev up-to-date: + bazel-bin/oppia_dev.aab INFO: Elapsed time: ... INFO: 1 process... INFO: Build completed successfully, ... ``` -Note also that the ``oppia.apk`` under the ``bazel-bin`` directory of your local copy of Oppia Android should be a fully functioning development version of the app that can be installed using ``adb`` +Note also that the ``oppia_dev.aab`` under the ``bazel-bin`` directory of your local copy of Oppia Android should be a fully functioning development version of the app that can be installed using bundle-tool. However, it's recommended to deploy Oppia to an emulator or connected device using the following Bazel command: + +```sh +bazel run //:install_oppia_dev +``` diff --git a/wiki/Bazel-Setup-Instructions-for-Windows.md b/wiki/Bazel-Setup-Instructions-for-Windows.md index 4200c5a8ead..287264e2331 100644 --- a/wiki/Bazel-Setup-Instructions-for-Windows.md +++ b/wiki/Bazel-Setup-Instructions-for-Windows.md @@ -51,63 +51,22 @@ sudo apt update && sudo apt upgrade After that, follow each of the subsections below as needed to install prerequisite dependencies: -**Java** +**Installing JDK 17+** -Bazel Setup requires both JDK 8 and JDK>=17 to complete the setup. +Setting up Bazel for Oppia Android requires JDK>=17 for [Android Package Manager](#3-installing-the-android-sdk). -- JDK >= 17 is required for [Android Package Manager](#3-installing-the-android-sdk). -- JDK 8 is required for the [Android build tools](#6-verifying-the-build), and we suggest installing OpenJDK. - -(Note: Our recommendation is to install both JDK 8 and JDK 17 and always make sure to run sdkmanager commands with JDK 17 and build commands with JDK 8. `sudo update-alternatives --config java` is used to set the default Java version). - -**Install JDK 8** - -```sh -sudo apt install openjdk-8-jdk-headless -``` - -**Install JDK 17** +For Ubuntu systems, this can be set up using: ```sh sudo apt install openjdk-17-jdk ``` -#### For Fedora 25+ - -- Install JDK 8 by running this command on the terminal: -``` -sudo dnf install java-1.8.0-openjdk -``` +For Fedora 25+, this can be set up using: -- Install JDK 17 by running this command on the terminal: ``` sudo dnf install java-17-openjdk ``` -#### Follow [these instructions](https://www.java.com/en/download/help/path.html) to correctly set up $JAVA_HOME. - -**Python 2** - -Unfortunately, some of the Bazel build actions in the Android pipeline require Python 2 to be installed: - -```sh -sudo apt install python -``` - -(This might throw an error - **E**: Package 'python' has no installation candidate) - -Alternatively, installl Python 2 using: - -```sh -sudo apt install python2 -``` - -To make python2 command accessible as python create a symbolic link from python2 to python: - -```sh -sudo ln -s /usr/bin/python2 /usr/bin/python -``` - **GCC** Install gcc using the following command: @@ -161,34 +120,23 @@ source ~/.bashrc (The last line reloads your Bash configuration file so that the variable adjustments above become live in your local terminal). -The ``sdkmanager`` command can now be used to install the necessary packages. - -**Set the default Java version to JDK-17** - -Prior to executing the sdkmanager commands, make sure to set the default Java version to jdk-17 by running the following command: - -```sh -sudo update-alternatives --config java -``` -Select the number with JDK-17. - -Run each of the following commands in succession (you may need to accept licenses for the SDK packages in the same way you would when using Android Studio): +The ``sdkmanager`` command can now be used to install the necessary packages. Run each of the following commands in succession (you may need to accept licenses for the SDK packages in the same way you would when using Android Studio): ```sh sdkmanager sdkmanager --install "platform-tools" sdkmanager --install "platforms;android-33" -sdkmanager --install "build-tools;29.0.2" +sdkmanager --install "build-tools;32.0.0" ``` When the commands above are finished running, the Android SDK should now be installed in your subsystem & be accessible to Bazel. ### 4. Installing Bazel -Follow [these instructions](https://docs.bazel.build/versions/main/install-ubuntu.html#install-on-ubuntu) to install Bazel using ``apt`` rather than Bazelisk (Bazelisk might work, but it's untested with these instructions). Note that Oppia requires Bazel 4.0.0, so you'll likely need to run the following command: +Follow [these instructions](https://docs.bazel.build/versions/main/install-ubuntu.html#install-on-ubuntu) to install Bazel using ``apt`` rather than Bazelisk (Bazelisk might work, but it's untested with these instructions). Note that Oppia requires Bazel 6.5.0, so you'll likely need to run the following command: ```sh -sudo apt install bazel-4.0.0 +sudo apt install bazel-6.5.0 ``` #### For Fedora 25+ @@ -215,8 +163,6 @@ Clone the [oppia-android](https://github.com/oppia/oppia-android) repository int git clone https://github.com/oppia/oppia-android.git ``` -From there, follow [these instructions](https://github.com/oppia/oppia-bazel-tools#readme) in order to prepare your environment to support Oppia Android builds. - To configure your development environment and set up essential tools, execute the following setup script from the oppia-android directory. ```sh @@ -227,19 +173,6 @@ scripts/setup.sh At this point, your system should be able to build Oppia Android. To verify, try building the APK (from your subsystem terminal -- note that this & all other Bazel commands must be run from the root of the ‘oppia-android’ directory otherwise they will fail): -To build, it is necessary to configure JDK 8 as the default. To accomplish this, follow these steps: - -**Set Default version to JDK 8** - -Prior to executing the build commands, make sure to set the default Java version to jdk-8 by running the following command: - -```sh -sudo update-alternatives --config java -``` - -Select the number with JDK-8 - -**Build** ```sh bazel build //:oppia ``` diff --git a/wiki/Installing-Oppia-Android.md b/wiki/Installing-Oppia-Android.md index 8e37749c5f6..ffc19cba8bd 100644 --- a/wiki/Installing-Oppia-Android.md +++ b/wiki/Installing-Oppia-Android.md @@ -14,10 +14,10 @@ This wiki page explains how to install Oppia Android on your local machine. If y ## Prepare developer environment -1. Download/Install [Android Studio Bumblebee | Patch 3](https://developer.android.com/studio/archive). - +1. Download/Install [Android Studio Bumblebee | Patch 3](https://developer.android.com/studio/archive). + **Note**: We recommend installing **Android Studio Bumblebee | 2021.1.1 Patch 3** because newer versions of Android Studio[ do not support running tests where shared source sets are used](https://issuetracker.google.com/issues/232007221#comment18), a configuration we use at Oppia. - + **Direct download Url**: [Windows](https://redirector.gvt1.com/edgedl/android/studio/install/2021.1.1.23/android-studio-2021.1.1.23-windows.exe) | [Linux](https://redirector.gvt1.com/edgedl/android/studio/ide-zips/2021.1.1.23/android-studio-2021.1.1.23-linux.tar.gz) | [Intel Mac](https://redirector.gvt1.com/edgedl/android/studio/install/2021.1.1.23/android-studio-2021.1.1.23-mac.dmg) | [Apple Silicon Mac](https://redirector.gvt1.com/edgedl/android/studio/install/2021.1.1.23/android-studio-2021.1.1.23-mac_arm.dmg) You can have multiple versions of Android Studio installed on your machine at the same time. @@ -25,10 +25,10 @@ This wiki page explains how to install Oppia Android on your local machine. If y 2. Configure your Android Studio - In Android Studio, open Tools > SDK Manager. - In the "SDK Platforms" tab (which is the default), select `API Level 28` and also `API Level 30` (for Bazel support). - - Also, navigate to the "SDK Tools" tab, click the "Show Package Details" checkbox at the bottom right, then click on "Android SDK Build-Tools 34-rc1" and select 29.0.2 (this is needed for Bazel support). + - Also, navigate to the "SDK Tools" tab, click the "Show Package Details" checkbox at the bottom right, then click on "Android SDK Build-Tools 34-rc1" and select 32.0.0 (this is needed for Bazel support). - Then, click "Apply" to download and install these two SDKs/Tools. - + - Must have **JDK 11** selected: - In Android Studio, open Settings > Build, Execution, Deployment > Build Tools > Gradle and edit the Gradle JDK field. @@ -55,8 +55,8 @@ Please follow these steps to set up Oppia Android on your local machine. - For Windows 1. Install [Git Bash Command Line](https://gitforwindows.org/) - - **Note**: When installing Git bash, check the option to add it to powershell, so that you can run bash commands from within powershell. + + **Note**: When installing Git bash, check the option to add it to powershell, so that you can run bash commands from within powershell. 2. Open Git Bash Command Line. 3. Navigate to `opensource\oppia-android\`. 4. Run the script `bash scripts/setup.sh`. @@ -134,7 +134,7 @@ We strongly recommend running tests on Robolectric which is faster because it do - For Instrumentation class, **org.oppia.android.testing.OppiaTestRunner**, will be selected by default. - For target, select the **Use the device/snapshot dropdown** option. - Verify that your setup looks like below: - + ![](https://user-images.githubusercontent.com/59600948/272657260-2e654891-61be-467a-8ebd-c997aa2abda6.png) - Finally, Click the "Apply" and "Okay" buttons. - You may need to repeat step (3) above to run the test with the new configuration. @@ -142,7 +142,7 @@ We strongly recommend running tests on Robolectric which is faster because it do - This configuration will run all the tests in that class. 5. To run only a specific test in a file: - Search or scroll down to the desired test name, to the left of the test name, click on the run icon and select **Run '`test name`''**. - + ### Next Steps - Congratulations, you are ready to work on your first issue! Take a look at our [good first issues](https://github.com/oppia/oppia-android/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22+no%3Aassignee) and leave a comment with your suggested fix. A maintainer will assign you the issue and provide any necessary guidance. diff --git a/wiki/Oppia-Bazel-Setup-Instructions.md b/wiki/Oppia-Bazel-Setup-Instructions.md index b4d962636de..b1c8609c519 100644 --- a/wiki/Oppia-Bazel-Setup-Instructions.md +++ b/wiki/Oppia-Bazel-Setup-Instructions.md @@ -16,7 +16,7 @@ Bazel is an open-source build and test tool similar to Make, Maven, and Gradle. ### Installation -1. Download and Install Java 8 using the links from the [Java website](https://www.java.com/en/download/). +1. Download and Install Java 11 using the links from the [Java website](https://www.java.com/en/download/). 2. **Select your Operating System for instructions on setting up Bazel:** diff --git a/wiki/Troubleshooting-Installation.md b/wiki/Troubleshooting-Installation.md index a05144fa2ed..f23f9280987 100644 --- a/wiki/Troubleshooting-Installation.md +++ b/wiki/Troubleshooting-Installation.md @@ -30,7 +30,7 @@ Here are some general troubleshooting tips for oppia-android. The specific platf ``` Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. - ``` + ``` then it's fine to ignore it. The message just appears to be a warning. We don't use Gradle 7.0, so this warning is fine to ignore. @@ -64,9 +64,9 @@ Here are some general troubleshooting tips for oppia-android. The specific platf 1. No matching toolchains (sdk_toolchain_type) ``` - ERROR: While resolving toolchains for target //:oppia: no matching toolchains found for types + ERROR: While resolving toolchains for target //:oppia: no matching toolchains found for types @bazel_tools//tools/android:sdk_toolchain_type - ERROR: Analysis of target '//:oppia' failed; build aborted: no matching toolchains found for types + ERROR: Analysis of target '//:oppia' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/android:sdk_toolchain_type INFO: Elapsed time: 12.805s INFO: 0 processes. @@ -76,18 +76,21 @@ Here are some general troubleshooting tips for oppia-android. The specific platf 2. If you encounter the following: - ``` - ERROR: While parsing option --override_repository=android_tools=~/oppia-bazel/android_tools: Repository - override directory must be an absolute path + external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.java:21: error: package com.android.dex does not exist + import com.android.dex.DexFormat; + ^ ``` - - Try to delete the `.bazelrc` file to solve the above error. + This means that you're still configured for using the custom Oppia Android tool repository for Bazel 4.x (which is no longer needed with Bazel 6.x+). You can fix this by updating your ~/.bazelrc file and either commenting out (e.g. by adding a ``#`` at the start of the line), or removing, the following line: + + ``` + build --override_repository=android_tools=/home/user/opensource/oppia-bazel-tools + ``` 3. **java.lang.ClassNotFoundException: com.android.tools.r8.compatdx.CompatDx** - If, when building the app binary, you encounter a failure that indicates that the CompatDx file cannot be found, this is likely due to you using a newer version of the Android build tools. You can manually downgrade to an older version of build-tools (particularly 29.0.2). Unfortunately, this can't be done through Android Studio but it can be done over a terminal. Follow the instructions listed [here](https://github.com/oppia/oppia-android/issues/3024#issuecomment-884513455) to downgrade your build tools & then try to build the app again. + If, when building the app binary, you encounter a failure that indicates that the CompatDx file cannot be found, this is likely due to you using a newer version of the Android build tools. You can manually downgrade to an older version of build-tools (particularly 32.0.0). Unfortunately, this can't be done through Android Studio but it can be done over a terminal. Follow the instructions listed [here](https://github.com/oppia/oppia-android/issues/3024#issuecomment-884513455) to downgrade your build tools & then try to build the app again. 4. If you encounter this error while building bazel in Mac M1: @@ -122,4 +125,4 @@ The `--noexperimental_check_desugar_deps` flag is explained in the [bazel blog]( ### Can’t find a particular issue? -If the error you get is not in the Troubleshooting section above, please create an issue providing all the necessary information and assign it to **@MohitGupta121**. +If the error you get is not in the Troubleshooting section above, please post a request for help on the team's discussions board for installation problems: https://github.com/oppia/oppia-android/discussions/categories/q-a-installation.