Skip to content

Commit

Permalink
Merge pull request #13 from daschott/user/daschott/ns_support
Browse files Browse the repository at this point in the history
* Added namespace filtering for pod-based commands (vfp-counter, capture)
* Print HCN v2 representation for HNS command, if available
* Reduced time and space complexity of vfp-counter & capture commands for better scalability
* Now honor reading in $KUBECONFIG environment variable
  • Loading branch information
daschott authored Sep 12, 2022
2 parents e658be9 + 9557d3a commit b148b1b
Show file tree
Hide file tree
Showing 36 changed files with 563 additions and 540 deletions.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: Bug report
about: Create a report to help us improve winspect
about: Create a report to help us improve wcnspect
title: ''
labels: ''
assignees: ''
Expand All @@ -11,7 +11,7 @@ assignees: ''
A clear and concise description of what the bug is.

**Component:**
Is the issue in the Winspect Client, the Server, or both?
Is the issue in the wcnspect Client, the Server, or both?

**To Reproduce**
Steps to reproduce the behavior.
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
name: Feature request
about: Suggest an idea for winspect
about: Suggest an idea for wcnspect
title: ''
labels: ''
assignees: ''

---

**Component (Winspect Client or Server):**
**Component (wcnspect Client or Server):**
Is this feature request related to the Client or the Server

**Is your feature request related to a problem? Please describe.**
Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Contribution to Winspect
# Contribution to wcnspect

You can contribute to Winspect with issues and PRs. Simply filing issues for problems you encounter is a great way to contribute. Contributing implementations is also greatly appreciated.
You can contribute to wcnspect with issues and PRs. Simply filing issues for problems you encounter is a great way to contribute. Contributing implementations is also greatly appreciated.

## Reporting Issues

Before filing a new issue, please search our [open issues](https://github.com/microsoft/winspect/issues) to check if it already exists.
Before filing a new issue, please search our [open issues](https://github.com/microsoft/wcnspect/issues) to check if it already exists.

If you do find an existing issue, please include your own feedback in the discussion. Do consider upvoting (👍 reaction) the original post, as this helps us prioritize popular issues in our backlog.

Expand Down
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
ifeq ($(OS),Windows_NT)
CLIENT_WINDOWS = set `GOARCH=amd64` && set `GOOS=windows` && go build -o out/bin/winspect.exe ./cmd/winspect/
CLIENT_LINUX = set `GOARCH=amd64` && set `GOOS=linux` && go build -o out/bin/winspect ./cmd/winspect/
CLIENT_WINDOWS = set `GOARCH=amd64` && set `GOOS=windows` && go build -o out/bin/wcnspect.exe ./cmd/wcnspect/
CLIENT_LINUX = set `GOARCH=amd64` && set `GOOS=linux` && go build -o out/bin/wcnspect ./cmd/wcnspect/

SERVER = set `GOARCH=amd64` && set `GOOS=windows` && go build -o out/bin/winspectserv.exe ./cmd/winspectserv/
SERVER = set `GOARCH=amd64` && set `GOOS=windows` && go build -o out/bin/wcnspectserv.exe ./cmd/wcnspectserv/

CLEAN = rmdir /S /q .\out
else
CLIENT_WINDOWS = GOARCH=amd64 GOOS=windows go build -o out/bin/winspect.exe ./cmd/winspect/
CLIENT_LINUX = GOARCH=amd64 GOOS=linux go build -o out/bin/winspect ./cmd/winspect/
CLIENT_WINDOWS = GOARCH=amd64 GOOS=windows go build -o out/bin/wcnspect.exe ./cmd/wcnspect/
CLIENT_LINUX = GOARCH=amd64 GOOS=linux go build -o out/bin/wcnspect ./cmd/wcnspect/

SERVER = GOARCH=amd64 GOOS=windows go build -o out/bin/winspectserv.exe ./cmd/winspectserv/
SERVER = GOARCH=amd64 GOOS=windows go build -o out/bin/wcnspectserv.exe ./cmd/wcnspectserv/

CLEAN = rm -rf out
endif
Expand Down
48 changes: 27 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Winspect - A Network Diagnostics Tool
# Wcnspect

> A network inspector for the Windows container networking stack on Kubernetes
> **W**indows **c**ontainer **n**etworking **s**tack in**spect**or
## Features

Currently, the winspect tool features four commands -
Currently, the wcnspect tool features four commands -

* Capture: runs a packet capture on Windows nodes, Has the capability to filter on pods, IPs, MACs, ports, protocols, and packet type (all, flow, or drop).
* Counter: will retrieve packet counter tables from windows nodes. It only outputs a table on nodes currently running a capture.
Expand Down Expand Up @@ -38,57 +38,59 @@ make server

It should be noted that while the client is cross-platform, the server can only run on Windows.

## Winspect Server
## Wcnspect Server

### Deploying the Winspect Server as a DaemonSet (Recommended)
### Deploying the wcnspect Server as a DaemonSet (Recommended)

You can apply the [winspectserv-daemon.yml](manifest/winspectserv-daemon.yml) to deploy the server as a host process container on all Windows nodes. The only way to do this, as of now, is for the user to create their own Azure Container Registry (ACR) and upload the image through there.
You can apply the [wcnspectserv-daemon.yml](manifest/wcnspectserv-daemon.yml) to deploy the server as a host process container on all Windows nodes. The only way to do this, as of now, is for the user to create their own Azure Container Registry (ACR) and upload the image through there.

The steps for using ACR are outlined <a href="https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tutorial-quick-task" target="_blank">in these docs.</a>
The [Dockerfile](manifest/Dockerfile) used for the original image is also contained in [manifest](./manifest).

Note that the [manifest](./manifest) directory also contains sample web server deployments for Windows Server 2019 and Windows Server 2022.


### Running Winspect Server directly
### Running wcnspect Server directly

If you decide not to deploy the server as a container and manually download it to a node, then it must be run with elevated Administrator permissions. You can run it as follows:

> running on the default port of 50051
```shell
./winspectserv
./wcnspectserv
```

> running on a chosen port of 43058
```shell
./winspectserv -p 43058
./wcnspectserv -p 43058
```
> **NOTE: The Winspect client only supports connecting to port 50051 currently**
> **NOTE: The wcnspect client only supports connecting to port 50051 currently**
## Winspect Client
## Wcnspect Client
The client needs to be executed as a standalone binary from either a Windows or a Linux VM in the same network (jumpbox).

The winspect client reads in the user's `.kube` config file and uses the `default` namespace.
The wcnspect client requires access to the [Kubernetes cluster config](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/).

By default, Wcnspect client will search for a file named `config` in the `$HOME/.kube` directory. Otherwise, it will use the $KUBECONFIG environment variable.

By default, most commands pull information from *all* Windows nodes.
Consequently, when using commands, the user should reference node names and pod names for better filtering of results.

> in-depth documentation and examples are available with the -h flag on any command
```shell
winspect -h
winspect capture -h
winspect hns all -h
wcnspect -h
wcnspect capture -h
wcnspect hns all -h
```

For commands that accept lists, input should be comma-separated and without spaces. For example, if we want to capture for 10 seconds on nodes named `win1`, `win2`, and `win3`, while also filtering only for TCP packets, we could do the following:

> sample capture command
```shell
winspect capture nodes win1,win2,win3 -t TCP -d 10
wcnspect capture nodes win1,win2,win3 -t TCP -d 10
```

The command will be routed to each node's internal IP on the cluster. It should be noted that if we don't pass a duration, the command will run indefinitely. Additionally, we can terminate the process on the referenced nodes at any time with `Ctrl+C`.
Expand All @@ -98,22 +100,26 @@ Note that if we pass the `--counters-only` flag to the `capture` command, then p
> sample capture command using --counters-only
```shell
winspect capture nodes win1 --counters-only
wcnspect capture nodes win1 --counters-only
```

Importantly, while the `vfp-counter` command runs on its own (given a pod), the `counter` command is tied to running instances of the `capture` command. Consequently, in order for it to output a table on any given node, a capture must be run on that node at the same time. The table will output packet counts tied to that capture.

```shell
winspect capture nodes win1,win3 -t TCP
winspect counter
wcnspect capture nodes win1,win3 -t TCP
wcnspect counter
```

## Assumptions

Currently, this project's code makes the following assumptions:

* The port that winspect server nodes use is 50051 (this is currently required on the client-side).
* When applying `winspectserv-daemon.yml`, the user has access to the ACR referenced in the file.
* The port that wcnspect server uses is 50051 (this is currently required on the client-side).
* When applying `wcnspectserv-daemon.yml`, the user has access to the ACR referenced in the file.

## TODO
* Support for other ports on the Wcnspect client.


## Contributing

Expand Down
4 changes: 2 additions & 2 deletions SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## How to file issues and get help

This project uses GitHub Issues to track bugs and feature requests. Please search the [open issues](https://github.com/microsoft/winspect/issues) before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a [new issue](https://github.com/microsoft/winspect/issues/new).
This project uses GitHub Issues to track bugs and feature requests. Please search the [open issues](https://github.com/microsoft/wcnspect/issues) before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a [new issue](https://github.com/microsoft/wcnspect/issues/new).

The Winspect project maintainers will respond to the best of their abilities on a best effort basis.
The wcnspect project maintainers will respond to the best of their abilities on a best effort basis.

For help and questions about using this project, please reach out to @daschott on the [#sig-windows](https://kubernetes.slack.com/messages/sig-windows) channel.

Expand Down
52 changes: 32 additions & 20 deletions cmd/winspect/cmd/capture.go → cmd/wcnspect/cmd/capture.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,11 @@ import (
"sync"
"syscall"

"github.com/microsoft/winspect/pkg/client"
"github.com/microsoft/winspect/pkg/k8spi"
pb "github.com/microsoft/winspect/rpc"
"github.com/microsoft/wcnspect/common"
"github.com/microsoft/wcnspect/pkg/client"
"github.com/microsoft/wcnspect/pkg/k8sapi"
pb "github.com/microsoft/wcnspect/rpc"
v1 "k8s.io/api/core/v1"

"github.com/spf13/cobra"
)
Expand All @@ -28,6 +30,7 @@ type captureCmd struct {

packetType string
countersOnly bool
namespace string

*baseBuilderCmd
}
Expand All @@ -39,14 +42,14 @@ func (b *commandsBuilder) newCaptureCmd() *captureCmd {
Use: "capture",
Short: "The 'capture' command will run a packet capture on all windows nodes.",
Long: `The 'capture' command will run a packet capture on all windows nodes. For example:
'winspect capture pods {pods} --protocols TCP -d 10'.`,
'wcnspect capture pods {pod1,pod2} --protocols TCP -d 10'.`,
}

captureTypes := []string{"all", "nodes", "pods"}
captureHelp := map[string]string{
"all": "Runs on all windows nodes in the AKS cluster.",
"nodes": "Specify which nodes winspect should send requests to using node names.",
"pods": "Specify which pods the capture should filter on. Supports up to two pod names. Automatically defines nodes to capture on.",
"nodes": "Specify which nodes wcnspect should send requests to using comma-separated node names.",
"pods": "Specify which pods the capture should filter on. Supports up to two comma-separated pod names.",
}
for _, name := range captureTypes {
subcmd := &cobra.Command{
Expand All @@ -56,10 +59,8 @@ func (b *commandsBuilder) newCaptureCmd() *captureCmd {
cc.printCapture(cmd.Name(), args)
},
}

cmd.AddCommand(subcmd)
}

cmd.PersistentFlags().Int32VarP(&cc.time, "time", "d", 0, "Time to run packet capture for (in seconds). Runs indefinitely given 0.")

cmd.PersistentFlags().StringSliceVarP(&cc.ips, "ips", "i", []string{}, "Match source or destination IP address. CIDR supported.")
Expand All @@ -69,43 +70,54 @@ func (b *commandsBuilder) newCaptureCmd() *captureCmd {

cmd.PersistentFlags().StringVar(&cc.packetType, "type", "all", "Select which packets to capture. Can be all, flow, or drop.")
cmd.PersistentFlags().BoolVar(&cc.countersOnly, "counters-only", false, "Collect packet counters only. No packet logging.")

cmd.PersistentFlags().StringVarP(&cc.namespace, "namespace", "n", common.DefaultNamespace, "Specify Kubernetes namespace to filter pods on.")
cc.baseBuilderCmd = b.newBuilderCmd(cmd)

return cc
}

func (cc *captureCmd) printCapture(subcmd string, endpoints []string) {
cc.validateArgs()

targetNodes := cc.getWinNodes()
var targetNodes []v1.Node
// Store mapping of NodeName => Pod IPs
hostMap := make(map[string][]string)

// Revise nodes and pods arguments based on command name
switch subcmd {
default:
targetNodes = cc.getWinNodes()
case "nodes":
if len(endpoints) == 0 {
log.Fatal("must pass node names when using 'winspect capture nodes ...'")
log.Fatal("must pass node names when using 'wcnspect capture nodes ...'")
}

nodes := strings.Split(endpoints[0], ",")
if err := client.ValidateNodes(nodes, cc.getWinNodeNames()); err != nil {
log.Fatal(err)
}

targetNodes = cc.getNodes(nodes)
case "pods":
// Use namespace command
//cc.cmd.PersistentFlags().StringVar(&cc.namespace, "namespace", common.DefaultNamespace, "Optionally specify Kubernetes namespace to filter pods on.")
if len(endpoints) == 0 {
log.Fatal("must pass pod names when using 'winspect capture pods ...'")
log.Fatal("must pass pod names when using 'wcnspect capture pods ...'")
}

pods := strings.Split(endpoints[0], ",")
if err := client.ValidatePods(pods, cc.getPodNames()); err != nil {
log.Fatal(err)
// Namespace
ns := k8sclient.GetNamespace(cc.namespace)
// Loop over Pod, Node
var p *v1.Pod
var nodeName string
for _, podName := range pods {
p = k8sclient.GetPod(podName, ns.GetName())
nodeName = p.Spec.NodeName
podIP := p.Status.PodIP
if nodeName != "" {
hostMap[nodeName] = append(hostMap[nodeName], podIP)
targetNodes = append(targetNodes, cc.getNode(nodeName))
}
}

hostMap = cc.getNodePodMap(pods)
targetNodes = cc.getPodsNodes(pods)
}

// Capture any sigint to send a StopCapture request
Expand All @@ -121,7 +133,7 @@ func (cc *captureCmd) printCapture(subcmd string, endpoints []string) {
for _, node := range targetNodes {
wg.Add(1)

name, ip := node.GetName(), k8spi.RetrieveInternalIP(node)
name, ip := node.GetName(), k8sapi.RetrieveInternalIP(node)

c, closeClient := client.CreateConnection(ip)
defer closeClient()
Expand Down
Loading

0 comments on commit b148b1b

Please sign in to comment.