Skip to content

OSPF Watcher is added #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions docs/telemetry/ospf-watcher.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# OSPF watcher. Tracking OSPF topology changes in Real-Time

![OSPF watcher containerlab](https://github.com/Vadims06/ospfwatcher/blob/4b5024e97cb87796ec8f9edf3d63abff0d5ba04a/containerlab/ospf01/ospfwatcher_containerlab.png)

This lab consists of 6 FRR routers and a single OSPF Watcher. Each router is pre-configured to be part of an OSPF domain with different network types. Any topology changes detected by the OSPF Watcher are logged in the file `watcher/watcher.log`. The logging capabilities can be enhanced with tools like ELK or Topolograph, enabling features such as searching changes by time and exporting data to message brokers, Zabbix, and more. For further details, refer to the Links page.

### OSPF Topology Watcher
OSPF Watcher is a monitoring tool of OSPF topology changes for network engineers. It works via passively listening to OSPF control plane messages through a specially established OSPF adjacency between OSPF Watcher and one of the network device. *It assists in analyzing incidents by determining the precise time and location of events, as well as their distribution across the network in a retrospective manner.*

#### Detected network events:
* OSPF neighbor adjacency Up/Down
* OSPF link cost changes
* OSPF networks appearance/disappearance from the topology

## Quickstart

1. [Install](https://containerlab.srlinux.dev/install/) containerlab.
2. Run the script to prepare environment:

```
./prepare.sh
```

3. Start the lab
```
sudo clab deploy --topo ospf01.clab.yml
```

4. Check that Watcher is ready (usually it requires 10-15sec to be ready). Proceed to the next step once Watcher is ready.
```
sudo docker logs clab-ospf01-ospf-watcher
```
Expected output:
```
lsdb_output:
OSPF Instance: 1


OSPF Router with ID (10.10.10.1)


Router Link States (Area 0.0.0.0)

LS age: 9
Options: 0x2 : *|-|-|-|-|-|E|-
LS Flags: 0x3
Flags: 0x0
LS Type:
OSPF LSDB has been received
Sniffing packets on interface: eth1
```

5. Start watching logs
```
sudo tail -f watcher/watcher.log
```

6. Change OSPF settings on lab' routers. Connect to a router
```
sudo docker exec -it clab-ospf01-router6 vtysh
```
Change metric on the interface
```
router6# conf t
router6(config)# int eth1
router6(config-if)# ip ospf cost 66
```

Add new stub network
```
router6(config-if)# ip address 10.10.36.6/24
```

Remove external type-2 subnet
```
router6(config-if)# exit
router6(config)# no ip route 6.6.6.6/32 192.168.36.3
```

Shutdown adjancency
```
router6(config)# int eth1
router6(config-if)# shutdown
```

### OSPF Watcher logs location
Available under `watcher` folder. To see them:
```
sudo tail -f watcher/watcher.log
```


##### Logs sample 1
```
2023-01-01T00:00:00Z,demo-watcher,host10.10.10.4,down,10.10.10.5,01Jan2023_00h00m00s_7_hosts,0,1234
```

* `2023-01-01T00:00:00Z` - event timestamp
* `demo-watcher` - name of watcher
* `host` - event name: `host`, `network`, `metric`
* `10.10.10.4` - event object. Watcher detected an event related to `10.10.10.4` host
* `down` - event status: `down`, `up`, `changed`
* `10.10.10.5` - event detected by this node.
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
* `0.0.0.0` - OSPF area ID
* `1234` - AS number where OSPF is working
*Summary: `10.10.10.5` detected that `10.10.10.4` host in area 0 in AS 1234 went down at `2023-01-01T00:00:00Z`*

##### Logs sample 2
```
2023-01-01T00:00:00Z,demo-watcher,network,192.168.13.0/24,changed,old_cost:10,new_cost:12,10.10.10.1,01Jan2023_00h00m00s_7_hosts,0.0.0.0,1234,internal,0
```

* `2023-01-01T00:00:00Z` - event timestamp
* `demo-watcher` - name of watcher
* `metric` - event name: `host`, `network`, `metric`
* `192.168.13.0/24` - event object. Watcher detected an event related to `192.168.13.0/24` subnet
* `changed` - event status: `down`, `up`, `changed`
* `10` - old cost
* `12` - new cost
* `10.10.10.1` - event detected by this node.
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
* `0.0.0.0` - OSPF area ID
* `1234` - AS number where OSPF is working
* `internal` - type of network: `internal` or `external`
* `0` - subtype of network: type-1, type-2 or 0 for internal subnets
*Summary: `10.10.10.1` detected that metric of `192.168.13.0/24` internal stub network changed from `10` to `12` at `2023-01-01T00:00:00Z` in area 0*


Note:
log file should have `systemd-network:systemd-journal` ownership

### Links

* [OSPF Watcher](https://github.com/Vadims06/ospfwatcher) - OSPF topology tracker
2 changes: 1 addition & 1 deletion labs/isis-watcher/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# IS-IS watcher. Tracking IS-IS topology changes in Real-Time

![IS-IS watcher containerlab](container_lab.drawio.png)
![IS-IS watcher containerlab](https://gitlab.com/rdodin/pics/-/wikis/uploads/d164f3d0688a85ed89206c25d7be4bb4/CleanShot_2025-03-22_at_13.11.19_2x.png)
This lab consists of 6 FRR routers and a single IS-IS Watcher. Each router is pre-configured to be part of an IS-IS domain with different network types. Any topology changes detected by the IS-IS Watcher are logged in the file `watcher/watcher.log`. The logging capabilities can be enhanced with tools like ELK or Topolograph, enabling features such as searching changes by time and exporting data to message brokers, Zabbix, and more. For further details, refer to the Links page.

### IS-IS Topology Watcher
Expand Down
131 changes: 131 additions & 0 deletions labs/ospf-watcher/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# OSPF watcher. Tracking OSPF topology changes in Real-Time

![OSPF watcher containerlab](ospfwatcher_containerlab.png)

This lab consists of 6 FRR routers and a single OSPF Watcher. Each router is pre-configured to be part of an OSPF domain with different network types. Any topology changes detected by the OSPF Watcher are logged in the file `watcher/watcher.log`. The logging capabilities can be enhanced with tools like ELK or Topolograph, enabling features such as searching changes by time and exporting data to message brokers, Zabbix, and more. For further details, refer to the Links page.

### OSPF Topology Watcher
OSPF Watcher is a monitoring tool of OSPF topology changes for network engineers. It works via passively listening to OSPF control plane messages through a specially established OSPF adjacency between OSPF Watcher and one of the network device. *It assists in analyzing incidents by determining the precise time and location of events, as well as their distribution across the network in a retrospective manner.*

#### Detected network events:
* OSPF neighbor adjacency Up/Down
* OSPF link cost changes
* OSPF networks appearance/disappearance from the topology

## Quickstart

1. [Install](https://containerlab.srlinux.dev/install/) containerlab.
2. Run the script to prepare environment:

```
./prepare.sh
```

3. Start the lab
```
sudo clab deploy --topo ospf01.clab.yml
```

4. Check that Watcher is ready (usually it requires 10-15sec to be ready). Proceed to the next step once Watcher is ready.
```
sudo docker logs clab-ospf01-ospf-watcher
```
Expected output:
```
lsdb_output:
OSPF Instance: 1


OSPF Router with ID (10.10.10.1)


Router Link States (Area 0.0.0.0)

LS age: 9
Options: 0x2 : *|-|-|-|-|-|E|-
LS Flags: 0x3
Flags: 0x0
LS Type:
OSPF LSDB has been received
Sniffing packets on interface: eth1
```

5. Start watching logs
```
sudo tail -f watcher/watcher.log
```

6. Change OSPF settings on lab' routers. Connect to a router
```
sudo docker exec -it clab-ospf01-router6 vtysh
```
Change metric on the interface
```
router6# conf t
router6(config)# int eth1
router6(config-if)# ip ospf cost 66
```

Add new stub network
```
router6(config-if)# ip address 10.10.36.6/24
```

Remove external type-2 subnet
```
router6(config-if)# exit
router6(config)# no ip route 6.6.6.6/32 192.168.36.3
```

Shutdown adjancency
```
router6(config)# int eth1
router6(config-if)# shutdown
```

### OSPF Watcher logs location
Available under `watcher` folder. To see them:
```
sudo tail -f watcher/watcher.log
```


##### Logs sample 1
```
2023-01-01T00:00:00Z,demo-watcher,host10.10.10.4,down,10.10.10.5,01Jan2023_00h00m00s_7_hosts,0,1234
```

* `2023-01-01T00:00:00Z` - event timestamp
* `demo-watcher` - name of watcher
* `host` - event name: `host`, `network`, `metric`
* `10.10.10.4` - event object. Watcher detected an event related to `10.10.10.4` host
* `down` - event status: `down`, `up`, `changed`
* `10.10.10.5` - event detected by this node.
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
* `0.0.0.0` - OSPF area ID
* `1234` - AS number where OSPF is working
*Summary: `10.10.10.5` detected that `10.10.10.4` host in area 0 in AS 1234 went down at `2023-01-01T00:00:00Z`*

##### Logs sample 2
```
2023-01-01T00:00:00Z,demo-watcher,network,192.168.13.0/24,changed,old_cost:10,new_cost:12,10.10.10.1,01Jan2023_00h00m00s_7_hosts,0.0.0.0,1234,internal,0
```

* `2023-01-01T00:00:00Z` - event timestamp
* `demo-watcher` - name of watcher
* `metric` - event name: `host`, `network`, `metric`
* `192.168.13.0/24` - event object. Watcher detected an event related to `192.168.13.0/24` subnet
* `changed` - event status: `down`, `up`, `changed`
* `10` - old cost
* `12` - new cost
* `10.10.10.1` - event detected by this node.
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
* `0.0.0.0` - OSPF area ID
* `1234` - AS number where OSPF is working
* `internal` - type of network: `internal` or `external`
* `0` - subtype of network: type-1, type-2 or 0 for internal subnets
*Summary: `10.10.10.1` detected that metric of `192.168.13.0/24` internal stub network changed from `10` to `12` at `2023-01-01T00:00:00Z` in area 0*


Note:
log file should have `systemd-network:systemd-journal` ownership
85 changes: 85 additions & 0 deletions labs/ospf-watcher/ospf01.clab.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: ospf01

topology:
nodes:
# note, that the bridge br-dr is created by prepare.sh
br-dr:
kind: bridge
ospf-watcher:
kind: linux
image: vadims06/ospf-watcher:latest
network-mode: container:router1
startup-delay: 30
stages:
create:
wait-for:
- node: router1
stage: create
- node: router2
stage: create
- node: router3
stage: create
- node: router4
stage: create
- node: router5
stage: create
- node: router6
stage: create
env:
WATCHER_LOGFILE: "/home/watcher/watcher/logs/watcher.log"
FRR_HOST: 127.0.0.1
FRR_PORT: 2604
FRR_PASSWORD: zebra
FRR_SECRET: zebra
ASN: '12345'
AREA_NUM: 0.0.0.0
WATCHER_NAME: ospfwatcher-demo
WATCHER_INTERFACE: eth1
binds:
- watcher/watcher.log:/home/watcher/watcher/logs/watcher.log
router1:
kind: linux
image: frrouting/frr:v8.4.1
binds:
- router1/daemons:/etc/frr/daemons
- router1/frr.conf:/etc/frr/frr.conf
router2:
kind: linux
image: frrouting/frr:v8.4.1
binds:
- router2/daemons:/etc/frr/daemons
- router2/frr.conf:/etc/frr/frr.conf
router3:
kind: linux
image: frrouting/frr:v8.4.1
binds:
- router3/daemons:/etc/frr/daemons
- router3/frr.conf:/etc/frr/frr.conf
router4:
kind: linux
image: frrouting/frr:v8.4.1
binds:
- router4/daemons:/etc/frr/daemons
- router4/frr.conf:/etc/frr/frr.conf
router5:
kind: linux
image: frrouting/frr:v8.4.1
binds:
- router5/daemons:/etc/frr/daemons
- router5/frr.conf:/etc/frr/frr.conf
router6:
kind: linux
image: frrouting/frr:v8.4.1
binds:
- router6/daemons:/etc/frr/daemons
- router6/frr.conf:/etc/frr/frr.conf


links:
- endpoints: ["router1:eth1", "router3:eth1"]
- endpoints: ["router1:eth5","mgmt-net:router1-5"]
- endpoints: ["router2:eth1", "router3:eth2"]
- endpoints: ["router3:eth4", "br-dr:eth13"]
- endpoints: ["router3:eth5", "router6:eth1"]
- endpoints: ["router4:eth1", "br-dr:eth14"]
- endpoints: ["router5:eth1", "br-dr:eth15"]
Binary file added labs/ospf-watcher/ospfwatcher_containerlab.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
14 changes: 14 additions & 0 deletions labs/ospf-watcher/prepare.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
if [ ! -d "watcher" ]; then
mkdir watcher
fi
if [ ! -f watcher/watcher.log ]; then
touch watcher/watcher.log
fi
# reset the log file to a clean slate
truncate -s0 watcher/watcher.log
sudo chown systemd-network:systemd-journal watcher/watcher.log
is_exist=$(brctl show br-dr)
if [[ -z "$is_exist" ]]; then
sudo brctl addbr br-dr && \
sudo ip link set up dev br-dr
fi
Loading