Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add [Coderabbit] PR Stats service and tests #10749

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions services/coderabbit/coderabbit-stats.service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import Joi from 'joi'
import { BaseJsonService } from '../index.js'

const schema = Joi.object({
reviews: Joi.number().required(),
}).required()

class CodeRabbitStats extends BaseJsonService {
static category = 'analysis'
static route = {
base: 'coderabbit',
pattern: 'stats/:provider/:org/:repo',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make the "noun" part of the route /prs or /pull-requests here. /stats doesn't really describe the information on the badge. It also leaves things more open if we ever wanted to add more CodeRabbit badges.
Docs on badge URLs: https://github.com/badges/shields/blob/master/doc/badge-urls.md

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chris48s this would be a major change on the endpoint side as we need to change the backend code. We are on a freeze now and can make this change in Jan 2025.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we are declaring here is the URL for the badge on shields.io

The route on the upstream API (i.e: https://api.coderabbit.ai/stats/... ) is independent of this.

}

static examples = [
chris48s marked this conversation as resolved.
Show resolved Hide resolved
{
title: 'CodeRabbit Review Stats',
namedParams: {
provider: 'github',
org: 'coderabbitai',
repo: 'ast-grep-essentials',
},
staticPreview: this.render({
reviews: 101,
}),
documentation: 'Shows the number of CodeRabbit reviews for a repository',
},
]

static defaultBadgeData = {
label: 'CodeRabbit',
labelColor: '171717',
}

static render({ reviews }) {
return {
message: `${reviews} Reviews`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From

https://github.com/badges/shields/blob/master/CONTRIBUTING.md#badge-guidelines

The left-hand side of a badge should not advertise. It should be a lowercase noun succinctly describing the meaning of the right-hand side.

So rather than CodeRabbit | 7 Reviews, this badge should be either reviews | 7 or coderabbit reviews | 7

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the text accordingly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The label text should be lowercase, even where we're including a service name

color: 'ff570a',
}
}

static renderError({ message }) {
return {
message,
color: '9f9f9f',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we ditch the custom colours here and stick with the standard palette please.
We can just get rid of the labelColor in defaultBadgeData. All badges should use the default #555555 for the label (left side of the badge)

For the message (right side), lets go with blue. We use this for informational badges in our palette, like this one where all badges get the same colour regardless of the number.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I do orange, which is one of the named colors? #fe7d37

https://github.com/badges/shields/tree/master/badge-maker#colors

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed colors fully

}
}

async fetch({ provider, org, repo }) {
return this._requestJson({
schema,
url: `https://api.coderabbit.ai/stats/${provider}/${org}/${repo}`,
httpErrors: {
404: 'invalid',
chris48s marked this conversation as resolved.
Show resolved Hide resolved
},
})
}

async handle({ provider, org, repo }) {
try {
const data = await this.fetch({ provider, org, repo })
return this.constructor.render(data)
} catch (error) {
return this.constructor.renderError({ message: error.message })
}
chris48s marked this conversation as resolved.
Show resolved Hide resolved
}
}

export default CodeRabbitStats
54 changes: 54 additions & 0 deletions services/coderabbit/coderabbit-stats.tester.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import { createServiceTester } from '../tester.js'

export const t = await createServiceTester()

t.create('live CodeRabbitStats')
.get('/stats/github/coderabbitai/ast-grep-essentials.json')
.expectBadge({
label: 'CodeRabbit',
message: /^\d+ Reviews$/, // Using regex pattern instead of isMetric
color: '#ff570a',
labelColor: '#171717',
})

t.create('CodeRabbitStats valid repo')
.get('/stats/github/coderabbitai/ast-grep-essentials.json')
.intercept(nock =>
nock('https://api.coderabbit.ai')
.get('/stats/github/coderabbitai/ast-grep-essentials')
.reply(200, { reviews: 101 }),
)
.expectBadge({
label: 'CodeRabbit',
message: '101 Reviews',
color: '#ff570a',
labelColor: '#171717',
})

t.create('CodeRabbitStats repo not found')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use live examples for all types of responses?

.get('/stats/github/not-valid/not-found.json')
.intercept(nock =>
nock('https://api.coderabbit.ai')
.get('/stats/github/not-valid/not-found')
.reply(404, 'invalid'),
)
.expectBadge({
label: 'CodeRabbit',
message: 'Not Found: invalid',
color: '#9f9f9f', // Note: without # prefix
labelColor: '#171717', // Note: without # prefix
})

t.create('CodeRabbitStats server error')
.get('/stats/github/coderabbitai/error-repo.json')
.intercept(nock =>
nock('https://api.coderabbit.ai')
.get('/stats/github/coderabbitai/error-repo')
.reply(500, 'Internal Server Error'),
)
.expectBadge({
label: 'CodeRabbit',
message: 'Inaccessible: Got status code 500 (expected 200)', // Match exact error message
color: '#9f9f9f',
labelColor: '#171717',
})
Loading