Skip to content

Adding Github MCP server and a new panel for querying the non-existent Github issues #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
knabben wants to merge 2 commits into
base: main
Choose a base branch
from
+1,113 −291
Open

Adding Github MCP server and a new panel for querying the non-existent Github issues #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f69d6fe
Adding Github MCP server and a new panel for querying the non-existent
knabben Nov 15, 2025
db17fdc
Adding flaky tests in the analysis
knabben Dec 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 60 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ Prerequisites

* Go 1.24 or later
* Git
* Github Token PAT
* Github Token PAT (required for GitHub integration and MCP server)
* Anthropic API Key (optional, required for AI-powered analysis in MCP client)
Copy link

@benjaminapetersen benjaminapetersen Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meaing a personal key, I forget if there is a cost to this or if you get some free usage via the API (rather than chat interface).

Copy link
Member Author

@knabben knabben Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, a personal key (as today is required for Github). https://www.claude.com/pricing#api I don't think the API allows free tier usage.

* Kubernetes cluster (kind)

### Prerequisites
Expand All @@ -58,6 +59,64 @@ export SIGNALHOUND_GITHUB_TOKEN=<github.pat.for.signalhound>
export GITHUB_TOKEN=<github.pat.default>
```

## Model Context Protocol (MCP) Integration

Signalhound includes a Model Context Protocol (MCP) integration that enables AI-powered analysis of failing tests and GitHub issues. The MCP system consists of two components:

### MCP Server

The MCP server runs automatically when you start the `abstract` command. It provides tools for querying GitHub project issues and is accessible at `http://localhost:8080/mcp` by default.

**Required Environment Variables:**
- `GITHUB_TOKEN` or `SIGNALHOUND_GITHUB_TOKEN` - GitHub Personal Access Token with `read:project` scope (required for the MCP server to query GitHub project issues)

**Features:**
- Lists all issues from the SIG Signal project board
- Filters issues by latest Kubernetes release version and FAILING status
- Returns issue details including number, title, body, state, and URL

### MCP Client

The MCP client is used by the TUI to analyze failing tests and compare them with existing GitHub issues using Anthropic's Claude AI.

**Required Environment Variables:**
- `ANTHROPIC_API_KEY` or `SIGNALHOUND_ANTHROPIC_API_KEY` - Anthropic API key for Claude AI analysis (required for AI-powered issue comparison)
- `MCP_SERVER_ENDPOINT` - MCP server endpoint URL (optional, defaults to `http://localhost:8080/mcp`)

**Features:**
- Compares currently failing tests with existing GitHub issues
- Identifies tests that don't have corresponding GitHub issues
- Provides AI-generated summaries and recommendations

### Complete Setup

To enable all MCP features, set the following environment variables:

```bash
# GitHub token for MCP server (to query project issues)
export GITHUB_TOKEN=<your-github-pat>
# or
export SIGNALHOUND_GITHUB_TOKEN=<your-github-pat>

# Anthropic API key for MCP client (for AI analysis)
export ANTHROPIC_API_KEY=<your-anthropic-api-key>
# or
export SIGNALHOUND_ANTHROPIC_API_KEY=<your-anthropic-api-key>

# Optional: Custom MCP server endpoint (defaults to http://localhost:8080/mcp)
export MCP_SERVER_ENDPOINT=http://localhost:8080/mcp
```

### Getting an Anthropic API Key

1. Visit [Anthropic's Console](https://console.anthropic.com/)
2. Sign up or log in to your account
3. Navigate to API Keys section
4. Create a new API key
5. Copy the key and set it as `ANTHROPIC_API_KEY` or `SIGNALHOUND_ANTHROPIC_API_KEY`

**Note:** The Anthropic API key is only required if you want to use the AI-powered analysis feature in the TUI. The MCP server will still work for listing issues without it, but the comparison and analysis features will not be available.

### Running at runtime

```bash
Expand Down
75 changes: 37 additions & 38 deletions cmd/abstract.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,21 @@
package cmd

import (
"context"
"fmt"
"log"
"net/http"
"os"
"time"

"github.com/spf13/cobra"

"sigs.k8s.io/signalhound/api/v1alpha1"
"sigs.k8s.io/signalhound/internal/mcp"
"sigs.k8s.io/signalhound/internal/testgrid"
"sigs.k8s.io/signalhound/internal/tui"
)

const MCP_SERVER = "localhost:8080"

// abstractCmd represents the abstract command
var abstractCmd = &cobra.Command{
Use: "abstract",
Expand All @@ -22,9 +26,7 @@ var abstractCmd = &cobra.Command{
}

var (
tg = testgrid.NewTestGrid(testgrid.URL)
minFailure, minFlake int
refreshInterval int
token string
)

Expand All @@ -35,50 +37,47 @@ func init() {
"minimum threshold for test failures, to disable use 0. Defaults to 0.")
abstractCmd.PersistentFlags().IntVarP(&minFlake, "min-flake", "m", 0,
"minimum threshold for test flakeness, to disable use 0. Defaults to 0.")
abstractCmd.PersistentFlags().IntVarP(&refreshInterval, "refresh-interval", "r", 0,
"refresh interval in seconds (0 to disable auto-refresh)")

token = os.Getenv("SIGNALHOUND_GITHUB_TOKEN")
if token == "" {
token = os.Getenv("GITHUB_TOKEN")
}
}

// FetchTabSummary fetches all dashboard tabs from TestGrid.
func FetchTabSummary() ([]*v1alpha1.DashboardTab, error) {
var dashboardTabs []*v1alpha1.DashboardTab
for _, dashboard := range []string{"sig-release-master-blocking", "sig-release-master-informing"} {
dashSummaries, err := tg.FetchTabSummary(dashboard, v1alpha1.ERROR_STATUSES)
if err != nil {
return nil, err
}
for _, dashSummary := range dashSummaries {
dashTab, err := tg.FetchTabTests(&dashSummary, minFailure, minFlake)
if err != nil {
fmt.Println(fmt.Errorf("error fetching table : %s", err))
continue
}
if len(dashTab.TestRuns) > 0 {
dashboardTabs = append(dashboardTabs, dashTab)
}
}
}
return dashboardTabs, nil
}

// RunAbstract starts the main command to scrape TestGrid.
func RunAbstract(cmd *cobra.Command, args []string) error {
dashboardTabs, err := FetchTabSummary()
if err != nil {
return err
}
var (
tg = testgrid.NewTestGrid(testgrid.URL)
dashboardTabs []*v1alpha1.DashboardTab
dashboards = []string{"sig-release-master-blocking", "sig-release-master-informing"}
)

var refreshFunc func() ([]*v1alpha1.DashboardTab, error)
if refreshInterval > 0 {
refreshFunc = func() ([]*v1alpha1.DashboardTab, error) {
return FetchTabSummary()
}
// start the MCP server in the background
go startMCPServer()

fmt.Println("Scraping the testgrid dashboard, wait...")
tuiInstance := tui.NewMultiWindowTUI(dashboardTabs, token)
tuiInstance.SetRefreshConfig(tg, dashboards, minFailure, minFlake)
return tuiInstance.Run()
}

// startMCPServer starts the MCP server in the background
func startMCPServer() {
ctx := context.Background()
mcpToken := os.Getenv("ANTHROPIC_API_KEY")
if mcpToken == "" {
log.Println("Warning: ANTHROPIC_API_KEY not set, MCP server will not start")
return
}
githubToken := os.Getenv("GITHUB_TOKEN")
if githubToken == "" {
log.Println("Warning: GITHUB_TOKEN not set, MCP server will not start")
return
}

return tui.RenderVisual(dashboardTabs, token, time.Duration(refreshInterval)*time.Second, refreshFunc)
server := mcp.NewMCPServer(ctx, githubToken)
log.Printf("MCP handler listening at %s", MCP_SERVER)
if err := http.ListenAndServe(MCP_SERVER, server.NewHTTPHandler()); err != nil {
log.Printf("MCP server error: %v", err)
}
}
26 changes: 16 additions & 10 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@ module sigs.k8s.io/signalhound
go 1.25.1

require (
github.com/anthropics/anthropic-sdk-go v1.19.0
github.com/gdamore/tcell/v2 v2.9.0
github.com/go-logr/logr v1.4.3
github.com/google/go-github/v65 v65.0.0
github.com/modelcontextprotocol/go-sdk v1.1.0
github.com/onsi/ginkgo/v2 v2.22.0
github.com/onsi/gomega v1.36.1
github.com/prometheus/client_golang v1.23.0
github.com/prometheus/common v0.65.0
github.com/rivo/tview v0.42.0
github.com/shurcooL/githubv4 v0.0.0-20240727222349-48295856cce7
github.com/spf13/cobra v1.8.1
Expand All @@ -18,9 +17,9 @@ require (
go.opentelemetry.io/otel/exporters/prometheus v0.60.0
go.opentelemetry.io/otel/metric v1.38.0
go.opentelemetry.io/otel/sdk/metric v1.38.0
golang.org/x/net v0.42.0
golang.org/x/net v0.46.0
golang.org/x/oauth2 v0.30.0
golang.org/x/text v0.28.0
golang.org/x/text v0.31.0
k8s.io/apimachinery v0.32.1
k8s.io/client-go v0.32.1
sigs.k8s.io/controller-runtime v0.20.2
Expand Down Expand Up @@ -53,8 +52,8 @@ require (
github.com/google/cel-go v0.22.0 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/jsonschema-go v0.3.0 // indirect
github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc // indirect
Expand All @@ -70,14 +69,21 @@ require (
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_golang v1.23.0 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.65.0 // indirect
github.com/prometheus/otlptranslator v0.0.2 // indirect
github.com/prometheus/procfs v0.17.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stoewer/go-strcase v1.3.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.28.0 // indirect
Expand All @@ -88,11 +94,11 @@ require (
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.35.0 // indirect
golang.org/x/term v0.34.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/time v0.7.0 // indirect
golang.org/x/tools v0.35.0 // indirect
golang.org/x/tools v0.38.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7 // indirect
Expand Down
Loading
Loading