Skip to content

Update analytics docs and add a page on PCW #1020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dadlerj merged 4 commits into from
Mar 7, 2025
Merged

Update analytics docs and add a page on PCW #1020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e223adf
Update analytics docs and add a page on PCW
dadlerj Mar 7, 2025
3e4df44
Add metric
dadlerj Mar 7, 2025
d4268e2
wip
dadlerj Mar 7, 2025
4fc7456
fix
dadlerj Mar 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 47 additions & 39 deletions docs/analytics/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ Self-hosted customers can also use this service, but they must

Upgrade to a supported version of Sourcegraph (5.9+)
Have fully enabled usage telemetry.
- [Enablement instructions](/analytics/cloud#enablement)

- [Enablement instructions](/analytics/cloud#enablement)

![Sourcegraph-cloud-analytics](https://storage.googleapis.com/sourcegraph-assets/Docs/Sourcegraph-Analytics-2025-01-28.png)

Expand All @@ -25,9 +26,16 @@ We provide a set of metrics that can be used to monitor Sourcegraph usage, engag

Learn more about how we think about [the ROI of Sourcegraph in our blog](https://sourcegraph.com/blog/measuring-the-impact-of-ai-coding-tools).

### User metrics available
### Overview metrics

| Metric | Description |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Percent of code written by Cody | Percentage of code written by Cody out of all code written during the selected time. [Learn more about this metric.](/analytics/pcw) |
| Lines of code written by Cody | Total lines of code written by Cody during the selected time |

### User metrics

| Metric | Description |
| Metric | Description |
| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Total active users | Total number of unique users who have used any Sourcegraph product during the selected time |
| Average daily users | The average number of unique users who use Sourcegraph per day during the selected time |
Expand All @@ -43,60 +51,60 @@ Cody-only user metrics

Many of the metrics above are also available for Cody only. However, some user definitions are slightly different:

| User category | Description |
| User category | Description |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Authenticated users | All users signed in to Cody in an editor or interacted with Cody on the web during the selected period. [Read more](/admin/pricing#authenticated-users). |
| Active users | All users who sign in to Cody in an editor or interact with Cody on the web during the period selected. [Read more](/admin/pricing#billable-active-users). |

### Code Search & navigation metrics

| Metric | Description |
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ |
| Total in-product searchers | Number of unique users who performed searches using the Sourcegraph search interface during the selected time |
| Total result clicks | Count of times users clicked on search results to view files or other resources during the selected time |
| Total file views | Number of times users viewed individual files through Sourcegraph during the selected time |
| Daily in-product search activity | Count of search operations performed each day through the Sourcegraph interface |
| Daily search users (in-product and API) | Number of unique users performing searches each day, including both UI and API usage |
| All searches (in-product and API) by type | Breakdown of searches by category (e.g., literal, regex), including both UI and API usage |
| Total code navigation actions | Count of all code navigation operations performed (e.g., go-to-definition, find references) during the selected time period |
| Precise code navigation % | Percentage of code navigation actions that used precise intelligence rather than search-based results during the selected time |
| Daily code navigation activity | Count of code navigation operations performed each day |
| Daily code navigation users | Number of unique users utilizing code navigation features each day |
| Precise vs. search-based code navigation actions by language | Comparison of precise vs. search-based navigation success rates broken down by programming language |
| Metric | Description |
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
| Total in-product searchers | Number of unique users who performed searches using the Sourcegraph search interface during the selected time |
| Total result clicks | Count of times users clicked on search results to view files or other resources during the selected time |
| Total file views | Number of times users viewed individual files through Sourcegraph during the selected time |
| Hours saved | The number of hours saved by search and code navigation users, assuming 5 minutes saved per search and 30 seconds per code navigation action |
| Daily in-product search activity | Count of search operations performed each day through the Sourcegraph interface |
| Daily search users (in-product and API) | Number of unique users performing searches each day, including both UI and API usage |
| All searches (in-product and API) by type | Breakdown of searches by category (e.g., literal, regex), including both UI and API usage |
| Total code navigation actions | Count of all code navigation operations performed (e.g., go-to-definition, find references) during the selected time period |
| Precise code navigation % | Percentage of code navigation actions that used precise intelligence rather than search-based results during the selected time |
| Daily code navigation activity | Count of code navigation operations performed each day |
| Daily code navigation users | Number of unique users utilizing code navigation features each day |
| Precise vs. search-based code navigation actions by language | Comparison of precise vs. search-based navigation success rates broken down by programming language |

### Autocompletion metrics

| Metric | Description |
| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Total accepted completions | Count of completions accepted by users during the selected time |
| Hours saved | The number of hours saved by Cody users, assuming 2 minutes saved per completion |
| Completions by day | The number of completions suggested by day and by editor. |
| Completion acceptance rate (CAR) | The percent of completions presented to a user for at least 750ms accepted by day, the editor, day, and month. |
| Metric | Description |
| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Total accepted completions | Count of completions accepted by users during the selected time |
| Hours saved | The number of hours saved by Cody users, assuming 2 minutes saved per completion |
| Completions by day | The number of completions suggested by day and by editor. |
| Completion acceptance rate (CAR) | The percent of completions presented to a user for at least 750ms accepted by day, the editor, day, and month. |
| Weighted completion acceptance rate (wCAR) | Similar to CAR, but weighted by the number of characters presented in the completion, by the editor, day, and month. This assigns more "weight" to accepted completions that provide more code to the user. |
| Completion persistence rate | Percent of completions that are retained or mostly retained (67%+ of inserted text) after various time intervals. |
| Average completion latency (ms) | The average milliseconds of latency before a user is presented with a completion suggestion by an editor. |
| Acceptance rate by language | CAR and total completion suggestions broken down by editor during the selected time |
| Completion persistence rate | Percent of completions that are retained or mostly retained (67%+ of inserted text) after various time intervals. |
| Average completion latency (ms) | The average milliseconds of latency before a user is presented with a completion suggestion by an editor. |
| Acceptance rate by language | CAR and total completion suggestions broken down by editor during the selected time |

### Chat and prompt metrics

| Metric | Description |
| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Total chat events | Total number of chat interactions with Cody during the selected time |
| Minutes saved per chat | Average time saved per chat interaction with Cody during the selected time |
| Hours saved by chats | Total hours saved through Cody chat interactions during the selected time , assuming 5 minutes saved per chat |
| Cody chats by day | Daily count of chat interactions |
| Cody chat users | Daily count of chat users |
| Lines of code inserted | Lines of code generated by Cody in chat that get applied, inserted, or pasted into the editor. Only VS Code is included in this metric for now |
| Insert rate | Percent of code generated by Cody in chat that gets applied, inserted, or pasted into the editor. Only VS Code is included in this metric for now |
| Chat apply & insert persistence rate | Percent of code inserted by Apply and Insert actions that are retained or mostly retained (67%+ of inserted text) after various time intervals |
| Prompts created, edited, and deleted by day | Daily count of prompt management activities, including creation, modification, and removal |
| Users creating, editing, and deleting prompts by day | Number of unique users performing prompt management activities each day |
| Metric | Description |
| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| Total chat events | Total number of chat interactions with Cody during the selected time |
| Hours saved by chats | Total hours saved through Cody chat interactions during the selected time , assuming 5 minutes saved per chat |
| Cody chats by day | Daily count of chat interactions |
| Cody chat users | Daily count of chat users |
| Lines of code inserted | Lines of code generated by Cody in chat that get applied, inserted, or pasted into the editor. Only VS Code is included in this metric for now |
| Insert rate | Percent of code generated by Cody in chat that gets applied, inserted, or pasted into the editor. Only VS Code is included in this metric for now |
| Chat apply & insert persistence rate | Percent of code inserted by Apply and Insert actions that are retained or mostly retained (67%+ of inserted text) after various time intervals |
| Prompts created, edited, and deleted by day | Daily count of prompt management activities, including creation, modification, and removal |
| Users creating, editing, and deleting prompts by day | Number of unique users performing prompt management activities each day |

### Command metrics (deprecated)

_As of Sourcegraph version 5.10, commands are being deprecated in favor of our new feature, chat prompts. As a result, if your Sourcegraph instance is running version 5.10 or later, you may see decreased command usage. In the coming weeks, these command metrics will be deprecated and replaced with new prompt metrics on the Chats tab._

| Metric | Description |
| Metric | Description |
| ----------------------- | ----------------------------------------------------------------------------------------------------------- |
| Total command events | Total number of command executions during the selected time |
| Hours saved by commands | Total hours saved through command executions during the selected time, assuming 5 minutes saved per command |
Expand Down
Loading