Tracking Contributions: Make Documentation team

[angelasjin]

Following some conversations with Milana (Documentation team rep), there are a few suggested contributions for tracking:

• In the Documentation team Github, track issues and assignments, possibly commenters.
• Tracking code examples or user contributed notes, both for the submitter and the reviewer

If possible, the team would like to be able to assign badges automatically for these types of contribution (in addition to being able to manually assign them.

[iandunn]

In the Documentation team Github, track issues and assignments, possibly commenters.

Opening/closing issues are already tracked (as of a few months ago), but I'm guessing comments would be easy. cc @dd32

Tracking code examples or user contributed notes, both for the submitter and the reviewer

👍🏻

[iandunn]

For GitHub activity to show up, though, folks need to connect their w.org accounts to their GitHub account.

Not everyone knows that, so maybe it's something we should make more visible somehow.

[courtneyr-dev]

Make visible

Lesson Plan about creating a .org account. We could elevate this to ALL teams for contributor days.

Would you like https://learn.wordpress.org/workshop/set-up-a-wordpress-org-account/ to cover it as well, or perhaps we can make a separate lesson plan and workshop about each?

We're working on courses around getting started as a contributor, and this would be super beginner-oriented. Folks should know some of this before even leaving issues in the forums.

[dd32]

When it comes to GitHub activity, I'd prefer to apply recorded events equally to all GitHub repo's, rather than specific rules for specific repo's / teams.

In the Documentation team Github, track issues and assignments, possibly commenters.

Opening/closing issues are already tracked (as of a few months ago), but I'm guessing comments would be easy.

Tracking Assignment of issues shouldn't be an issue, I'm hesitant to track Comments though due to the sheer number of comments made.

Perhaps this is one of the locations where comments tracking would be worthwhile though?

[iandunn]

I'm hesitant to track Comments though due to the sheer number.

That's also going to be a problem when we start tracking translations. One idea for that is to add something like Wrote 15 comments in 3 GitHub repositories this week, rather than 15 individual activity items. GitHub might provide a permalink we could include.

Or maybe we add individual items in the database, but then on the front end we display them all with something like <details>, for a better UX:

<details>
	<summary>Wrote 15 comments in 3 GitHub repositories this week</summary>
	
	<ul>
		<li>Wrote a comment on issue #371 in the WordPress/wporg-news-2021 repository:
			lorum ipsum blah blah blah
		</li>

		<li>Wrote a comment on issue #5432 in the WordPress/gutenberg repository:
			lorum ipsum blah blah blah
		</li>

		<li>Wrote a comment on issue #2353 in the WordPress/Documentation-Issue-Tracker repository:
			lorum ipsum blah blah blah
		</li>
	</ul>
</details>

We add an individual item for forum replies and p2 comments, though, which seems analogous. Are GH/Trac comments significantly more frequent?

[iandunn]

Would you like https://learn.wordpress.org/workshop/set-up-a-wordpress-org-account/ to cover it as well, or perhaps we can make a separate lesson plan and workshop about each?

I don't have a strong opinion about how, but I think mentioning it is a great idea 👍🏻

[zzap]

I'm hesitant to track Comments though due to the sheer number of comments made.

Fighting with all the notifications every day, I 100% agree with this. On the other hand, Documentation Issues Tracker repo has little to no other activity. We use that repo only for tracking issues thus comments being almost the only contribution.

We also saw a lot of contributions hunting for badges in past and that's the reason why I'm not for automating tracking contributions just by posting a comment. Some kind of manual action here would be appreciated even though nothing as a possible solution comes to mind.

In a perfect world we would have some kind of Team dashboard at wp.org, where we could see all tracked contributions from all places and a button "Confirm" next to each 😍

[dd32]

Or maybe we add individual items in the database, but then on the front end we display them all with something like <details>, for a better UX:

That seems like a reasonable idea to me, it'll require some pretty severe changes to the Profiles querying method though.

We add an individual item for forum replies and p2 comments, though, which seems analogous. Are GH/Trac comments significantly more frequent?

I don't get the feeling that P2s are all that active on WordPress.org unfortunately, make.wordpress.org/core would probably be the most active one I would've assume, but there's only been 127 comments by 89 people since 2021-04-01 (26 days), 3-5 entries per day in that period.

In the same time, we've had ~2.5k GitHub activity entries, and that's just from Issues & PRs open/closing.

[iandunn]

it'll require some pretty severe changes to the Profiles querying method though.

I was thinking something like:

• Start inserting comments into wporg_github_activity
• Add an index on the category column
• Add a WHERE category <> 'comment' to the query in WPORG_Profiles::get_code()
• Add a cron job that runs once a week and:
  • Selects all comment items in the past week
  • Loops through the query results and re-indexes them by user (i.e., $comments[ $user ][] = $comment;
  • Loops through the indexed array and adds a single activity entry with the <details> markup for each user
• Modify WPORG_Profiles::get_activities() to not truncate content when it contains <details>

Do you think it'd be more complicated than that?

A similar approach could be used for something like Wrote 15 comments in 3 GitHub repositories this week, if we don't want the details.

[dd32]

Do you think it'd be more complicated than that?

Well, I was thinking that you'd want the activity items to show instantly, not be delayed by a week.. I didn't really think of adding it to a cron instead. I still don't think I like the cron approach.

My complicated thing was that I figured we'd just be querying in realtime, not cron'ing it, and in that case the 50 limit and max 10 items logic in the activity stream would result in yesterdays items falling off the screen very quickly, even though it's including last weeks activity for other items.

[iandunn]

I was thinking that you'd want the activity items to show instantly, not be delayed by a week.

Fair point. We could create a rolling digest, instead of a static one.

• User foo hasn't posted any comments this week, and doesn't have any comment activity for this week on their profile
• They post a comment
• A cron is running every 15 minutes, and it sees the new comment. Since there aren't any others this week, it creates an activity entry in the BuddyPress database.
• Later they post a 2nd comment
• The next time the cron runs, it updates the existing activity entry, which now says they've posted 2 comments this week

[StevenDufresne]

In the Documentation team Github, track issues and assignments, possibly commenters.

We are currently not tracking assignments, though as @dd32 has mentioned it's easy to turn on. But, if we do track comments, and it appears as we will.... Are assignments still useful? You can be assigned by someone else but never end up doing anything.

[StevenDufresne]

Summarizing where I think this ticket is:

Tracking code examples or user-contributed notes, both for the submitter and the reviewer

Will do, not implement

Do we need assignments?

Needs Decision

Should we add comments?

Needs Decision

• Yes
  • How do we reduce noise?
    • Cron Job
  • How do we limit badge hunting?

I tend to agree that GitHub activity should be shared across all teams/repos and should it not work for some teams, we could exclude them. If that is a fair assumption, could we move the comments discussion & assignments to separate ( or the same ) tickets? That would leave us with a clearer scope on this ticket and we could action the first item which is documentation team specific.

[iandunn]

if we do track comments .... Are assignments still useful?

I would guess that they're not as important, since the comments are the actual contribution. Or are assignments itself a significant contribution, since it represents managing & delegating workflow on the Docs team; @zzap ?

How do we reduce noise?

#197 will solve that

How do we limit badge hunting

The scope of this milestone focuses on activity rather than badges, so I think we can leave that discussion to another day

move the comments discussion & assignments to separate ( or the same ) tickets?

Either that, or add checklists to this ticket. I don't have an opinion as long as it's obvious what's been done and what hasn't

[StevenDufresne]

Tracking code examples or user-contributed notes, both for the submitter and the reviewer
Partially implement in https://meta.trac.wordpress.org/changeset/11934.

This will turn on comments for developer.wordpress.org. However, there are no code paths that credit the reviewer at the moment. Do we want to turn that on for all reviewers? It seems like that type of action should be credited consistently.

[iandunn]

Tracking reviews seems like a good idea. It might be something we want to use digests for (#197). The current code doesn't handle bulk actions yet (like approving 20 translations, or 20 devhub comments), though. I'm working on that for #196. It may be good to wait and see how that turns out.

[iandunn]

The bulk process is working now:

Notifier: WordPress/wordpress.org@fbade87
Handler: WordPress/wordpress.org@564088b

[zzap]

if we do track comments .... Are assignments still useful?

I would guess that they're not as important, since the comments are the actual contribution. Or are assignments itself a significant contribution, since it represents managing & delegating workflow on the Docs team; @zzap ?

I'm struggling with that one to be honest. Assignments are significant but sometimes we want people who comment to get attribution as well. But not for all comments, especially because that's an easy way for badge hunting.

If it's possible to track it the same way Trac does - certain people comment props to... and that gets tracked, that would be ideal. If not, I guess we'll have to do it manually.