Skip to content

Commit

Permalink
Fix NPS Survey Gating (#5356)
Browse files Browse the repository at this point in the history
<!-- READ ME FIRST: Please fill in the explanation section below and
check off every point from the Essential Checklist! -->
## Explanation
I found out during testing that there is an underlying gating
DataProvider update behavior that occurs because of the way I initially
designed the survey popup behaviour:
- when the survey popup is displayed, we update the user profile with
the current survey shown timestamp, to be used as gating so that the
survey isn't triggered again before the grace period ends.
- Updating the profile notifies upstream observers, including the gating
DataProvider.
- Due to this update, the gating provider is triggered to recompute.
- This time, the gating result will be false for triggering the survey,
since the grace period condition has changed.
- The UI behavior is to navigate away from the exploration screen if the
survey isn't to be shown. The activity is destroyed, and the popup along
with it.

I mitigated this by removing the observer once the initial gating result
has been received and processed.

## Tests
I added tests that test survey gating based on changes in the time
criterion, while other gating criteria remains constant/fulfilled.
Exhaustive gating tests were added in the SurveyGatingController.
 
I also made a change in the `StateFragmentTest` that unregisters the
idling resource after initializing profiles instead of at the end of the
test runs. Previously, `StateFragmentTest`would not run locally and fail
with the error:

```
Error performing 'single click - At Coordinates: 173, 329 and precision: 16, 16' on view 'with id: org.oppia.android:id/play_test_exploration_button'.
at androidx.test.espresso.PerformException$Builder.build(PerformException.java:86)
at androidx.test.espresso.base.DefaultFailureHandler.getUserFriendlyError(DefaultFailureHandler.java:87)
at androidx.test.espresso.base.DefaultFailureHandler.handle(DefaultFailureHandler.java:59)
at androidx.test.espresso.ViewInteraction.waitForAndHandleInteractionResults(ViewInteraction.java:322)
at androidx.test.espresso.ViewInteraction.desugaredPerform(ViewInteraction.java:178)
at androidx.test.espresso.ViewInteraction.perform(ViewInteraction.java:119)
at org.oppia.android.app.player.state.StateFragmentTest.startPlayingExploration(StateFragmentTest.kt:475)org.oppia.android.app.player.state.StateFragmentTest.testFinishChapter_lateNight_isPastGracePeriod_minimumAggregateTimeMet_noSurveyPopup(StateFragmentTest.kt:264)
Caused by: androidx.test.espresso.IdlingResourceTimeoutException: Wait for [TestCoroutineDispatcherIdlingResource] to become idle timed out
at androidx.test.espresso.IdlingPolicy.handleTimeout(IdlingPolicy.java:61)
at androidx.test.espresso.base.UiControllerImpl$5.resourcesHaveTimedOut(UiControllerImpl.java:434)
at androidx.test.espresso.base.IdlingResourceRegistry$Dispatcher.handleTimeout(IdlingResourceRegistry.java:481)
```

This was caused by registering an idling resource before test begins,
then when the activity starts, the main thread becomes busy. Generally,
the test will continue when the main thread becomes idle, but the idling
resource now blocks it and does not call onTransitionToIdle(). So it
will always time out. Unregistering the idling resource before the
activity starts frees up the main thread.

Disabled running on Robolectric due to [known
issue](#1612) with Drag and
Drop interaction.

Espresso Run Screenshot
![Screenshot 2024-03-13 at 19 35
12](https://github.com/oppia/oppia-android/assets/59600948/72d8aee9-f39d-4d26-92a9-72fb0919d46c)

## Essential Checklist
<!-- Please tick the relevant boxes by putting an "x" in them. -->
- [ ] The PR title and explanation each start with "Fix #bugnum: " (If
this PR fixes part of an issue, prefix the title with "Fix part of
#bugnum: ...".)
- [x] Any changes to
[scripts/assets](https://github.com/oppia/oppia-android/tree/develop/scripts/assets)
files have their rationale included in the PR explanation.
- [x] The PR follows the [style
guide](https://github.com/oppia/oppia-android/wiki/Coding-style-guide).
- [x] The PR does not contain any unnecessary code changes from Android
Studio
([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#undo-unnecessary-changes)).
- [x] The PR is made from a branch that's **not** called "develop" and
is up-to-date with "develop".
- [x] The PR is **assigned** to the appropriate reviewers
([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#clarification-regarding-assignees-and-reviewers-section)).

## For UI-specific PRs only
<!-- Delete these section if this PR does not include UI-related
changes. -->
If your PR includes UI-related changes, then:
- Add screenshots for portrait/landscape for both a tablet & phone of
the before & after UI changes
- For the screenshots above, include both English and pseudo-localized
(RTL) screenshots (see [RTL
guide](https://github.com/oppia/oppia-android/wiki/RTL-Guidelines))
- Add a video showing the full UX flow with a screen reader enabled (see
[accessibility
guide](https://github.com/oppia/oppia-android/wiki/Accessibility-A11y-Guide))
- For PRs introducing new UI elements or color changes, both light and
dark mode screenshots must be included
- Add a screenshot demonstrating that you ran affected Espresso tests
locally & that they're passing
  • Loading branch information
adhiamboperes authored Apr 6, 2024
1 parent f0e9ac9 commit 9e8553c
Show file tree
Hide file tree
Showing 10 changed files with 709 additions and 95 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import androidx.appcompat.widget.Toolbar
import androidx.core.view.doOnPreDraw
import androidx.databinding.DataBindingUtil
import androidx.lifecycle.LiveData
import androidx.lifecycle.Observer
import androidx.lifecycle.Transformations
import org.oppia.android.R
import org.oppia.android.app.activity.ActivityScope
Expand Down Expand Up @@ -297,11 +298,7 @@ class ExplorationActivityPresenter @Inject constructor(
}
is AsyncResult.Success -> {
oppiaLogger.d("ExplorationActivity", "Successfully stopped exploration")
if (isCompletion) {
maybeShowSurveyDialog(profileId, topicId)
} else {
backPressActivitySelector()
}
maybeShowSurveyDialog(profileId, topicId)
}
}
}
Expand Down Expand Up @@ -528,42 +525,48 @@ class ExplorationActivityPresenter @Inject constructor(
}

private fun maybeShowSurveyDialog(profileId: ProfileId, topicId: String) {
surveyGatingController.maybeShowSurvey(profileId, topicId).toLiveData()
.observe(
activity
) { gatingResult ->
when (gatingResult) {
is AsyncResult.Pending -> {
oppiaLogger.d("ExplorationActivity", "A gating decision is pending")
}
is AsyncResult.Failure -> {
oppiaLogger.e(
"ExplorationActivity",
"Failed to retrieve gating decision",
gatingResult.error
)
backPressActivitySelector()
}
is AsyncResult.Success -> {
if (gatingResult.value) {
val dialogFragment =
SurveyWelcomeDialogFragment.newInstance(
profileId,
topicId,
explorationId,
SURVEY_QUESTIONS
)
val transaction = activity.supportFragmentManager.beginTransaction()
transaction
.add(dialogFragment, TAG_SURVEY_WELCOME_DIALOG)
.addToBackStack(null)
.commit()
} else {
val liveData = surveyGatingController.maybeShowSurvey(profileId, topicId).toLiveData()
liveData.observe(
activity,
object : Observer<AsyncResult<Boolean>> {
override fun onChanged(gatingResult: AsyncResult<Boolean>?) {
when (gatingResult) {
is AsyncResult.Pending -> {
oppiaLogger.d("ExplorationActivity", "A gating decision is pending")
}
is AsyncResult.Failure -> {
oppiaLogger.e(
"ExplorationActivity",
"Failed to retrieve gating decision",
gatingResult.error
)
backPressActivitySelector()
}
is AsyncResult.Success -> {
if (gatingResult.value) {
val dialogFragment =
SurveyWelcomeDialogFragment.newInstance(
profileId,
topicId,
explorationId,
SURVEY_QUESTIONS
)
val transaction = activity.supportFragmentManager.beginTransaction()
transaction
.add(dialogFragment, TAG_SURVEY_WELCOME_DIALOG)
.addToBackStack(null)
.commit()

// Changes to underlying DataProviders will update the gating result.
liveData.removeObserver(this)
} else {
backPressActivitySelector()
}
}
}
}
}
)
}

companion object {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,6 @@ class StateFragmentPresenter @Inject constructor(
fun onReturnToTopicButtonClicked() {
hideKeyboard()
markExplorationCompleted()
maybeShowSurveyDialog(profileId, topicId)
}

private fun showOrHideAudioByState(state: State) {
Expand Down Expand Up @@ -455,13 +454,17 @@ class StateFragmentPresenter @Inject constructor(
fun getExplorationCheckpointState() = explorationCheckpointState

private fun markExplorationCompleted() {
storyProgressController.recordCompletedChapter(
val markStoryCompletedLivedata = storyProgressController.recordCompletedChapter(
profileId,
topicId,
storyId,
explorationId,
oppiaClock.getCurrentTimeMs()
)
).toLiveData()

// Only check gating result when the previous operation has completed because gating depends on
// result of saving the time spent in the exploration, at the end of the exploration.
markStoryCompletedLivedata.observe(fragment, { maybeShowSurveyDialog(profileId, topicId) })
}

private fun showHintsAndSolutions(helpIndex: HelpIndex, isCurrentStatePendingState: Boolean) {
Expand Down Expand Up @@ -535,44 +538,43 @@ class StateFragmentPresenter @Inject constructor(
}

private fun maybeShowSurveyDialog(profileId: ProfileId, topicId: String) {
surveyGatingController.maybeShowSurvey(profileId, topicId).toLiveData()
.observe(
activity,
{ gatingResult ->
when (gatingResult) {
is AsyncResult.Pending -> {
oppiaLogger.d("StateFragment", "A gating decision is pending")
}
is AsyncResult.Failure -> {
oppiaLogger.e(
"StateFragment",
"Failed to retrieve gating decision",
gatingResult.error
)
surveyGatingController.maybeShowSurvey(profileId, topicId).toLiveData().observe(
activity,
{ gatingResult ->
when (gatingResult) {
is AsyncResult.Pending -> {
oppiaLogger.d("StateFragment", "A gating decision is pending")
}
is AsyncResult.Failure -> {
oppiaLogger.e(
"StateFragment",
"Failed to retrieve gating decision",
gatingResult.error
)
(activity as StopStatePlayingSessionWithSavedProgressListener)
.deleteCurrentProgressAndStopSession(isCompletion = true)
}
is AsyncResult.Success -> {
if (gatingResult.value) {
val dialogFragment =
SurveyWelcomeDialogFragment.newInstance(
profileId,
topicId,
explorationId,
SURVEY_QUESTIONS
)
val transaction = activity.supportFragmentManager.beginTransaction()
transaction
.add(dialogFragment, TAG_SURVEY_WELCOME_DIALOG)
.commitNow()
} else {
(activity as StopStatePlayingSessionWithSavedProgressListener)
.deleteCurrentProgressAndStopSession(isCompletion = true)
}
is AsyncResult.Success -> {
if (gatingResult.value) {
val dialogFragment =
SurveyWelcomeDialogFragment.newInstance(
profileId,
topicId,
explorationId,
SURVEY_QUESTIONS
)
val transaction = activity.supportFragmentManager.beginTransaction()
transaction
.add(dialogFragment, TAG_SURVEY_WELCOME_DIALOG)
.commitNow()
} else {
(activity as StopStatePlayingSessionWithSavedProgressListener)
.deleteCurrentProgressAndStopSession(isCompletion = true)
}
}
}
}
)
}
)
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ package org.oppia.android.app.player.state.testing
import android.widget.Button
import androidx.appcompat.app.AppCompatActivity
import androidx.databinding.DataBindingUtil
import androidx.lifecycle.Observer
import org.oppia.android.R
import org.oppia.android.app.activity.ActivityScope
import org.oppia.android.app.model.ProfileId
Expand Down Expand Up @@ -98,7 +97,7 @@ class StateFragmentTestActivityPresenter @Inject constructor(
}
startPlayingProvider.toLiveData().observe(
activity,
Observer<AsyncResult<Any?>> { result ->
{ result ->
when (result) {
is AsyncResult.Pending -> oppiaLogger.d(TEST_ACTIVITY_TAG, "Loading exploration")
is AsyncResult.Failure ->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ class SurveyWelcomeDialogFragmentPresenter @Inject constructor(
}

profileManagementController.updateSurveyLastShownTimestamp(profileId)

logSurveyPopUpShownEvent(explorationId, topicId, profileId)

return binding.root
Expand Down
Loading

0 comments on commit 9e8553c

Please sign in to comment.