Already_seen banner appearing too frequently

[wgranger]

Per Oct 24 Slack convo.

There may be an issue with how often the already seen banner is showing up and also how subjects are added to the queue.

After saving a classification, it looks like a subject is marked already_seen in the subjectList here -

mobile/src/reducers/classifierReducer.js

Line 74 in f975481

return R.set(R.lensProp('already_seen'), true, subject)

And then, the first subject that isn’t marked already_seen is grabbed here and made the next subject -

mobile/src/reducers/classifierReducer.js

Line 100 in f975481

const subject = usableSubjects[0]

But, if all subjects are marked already_seen in the above line, a random subject from the list is grabbed, which has probably already been seen. That might explain why already seen subjects are seen too frequently.

Later, if there are less than five subjects in the list that haven’t been seen, new subjects are grabbed here from the /queued route. However, those subjects are tacked on to the existing list without changing the fact that the current one might’ve already been seen -

mobile/src/actions/classifier.js

Line 114 in a8ca98f

dispatch(addSubjectsForWorklow(workflow.id))

I think the solution would be to get new subjects from the queued route here instead of a random subject

mobile/src/reducers/classifierReducer.js

Line 104 in f975481

// If the user has seen every subject, display a random one they have seen

and remove already seen subjects from the workflow subjectList, because I’m not sure why they would need to be kept in the list if they’ve already been seen.

[camallen]

linked to #289

That issue says:
Originally reported via https://www.zooniverse.org/talk/739/1153260

All subjects are marked with already seen banners once the first 4/5 subjects are classified.

This only impacts on workflows with configuration directive subject_queue_page_size = 4. I.e. science scribbler team reported this. It might only happen on swipe card workflows as well.

Noting that i've checked the selection APIs and they aren't returning seen images in any subject requests.

I was using Zooniverse App v2.6.2

1. Using Android 7.0
2. Moto G4 plus
3. Navigate to science scribbler (or similarly configured workflow) and swipe classify some images
4. I get new subjects not seen images
5. The list of subjects is a repeat of what i've recently classified
6. Science scribbler mobile workflow, though it should happen on any workflow with workflow.configuration['subject_queue_page_size'] = 4
7. Logged in user and logged out.

[lcjohnso]

@wgranger @camallen There's a ticket reporting similar "Already Seen" behavior, but for a project and workflow (Snapshot Enonkishu; project ID 8676, workflow ID 12460) that did not have a custom subject_queue_page_size.

[wgranger]

@lcjohnso Thanks for the heads up. I believe they could still erroneously see the "Already Seen" banner, regardless of the subject_queue_page_size, based on the logic above.

The other issue they bring up in their ticket is a separate bug that we have dealt with recently, but a new issue should be drawn up to address it.

[camallen]

I've had reports of this happening on fast workflows for science scribbler, https://www.zooniverse.org/projects/msbrhonclif/science-scribbler/talk/1147/1428346?comment=2321278&page=1
https://www.zooniverse.org/projects/msbrhonclif/science-scribbler/talk/1150/1406546?comment=2322522&page=2

I looked into the users and they have a lot of data remaining to classify on these workflows. I requested the user try the desktop app (so hitting the same selector services) but different client side handling of the incoming set of subjects. They saw no seen banners on the PFE client but did on the mobile client.

If we're going to continue to launch mobile workflows we have to spend some time investigating this to avoid wasting volunteer effort.

[lcjohnso]

Perhaps also related: #287