[Bug] VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo. #358

n-rodriguez · 2024-11-28T01:04:07Z

Is this urgent?

Yes, it's critical

How are you running netbox-ssot?

From source (Go)

What is the version of netbox-ssot

Running version unknown built on unknown (commit unknown)

Which module has the issue?

General (not module-specific)

Share your configuration

---
logger:
  level: 3
  dest: ""

netbox:
  apiToken: "foo"
  hostname: "netbox.foo.corp"
  httpScheme: http
  port: 80
  timeout: 30
  sourcePriority: ["vcenter67.corp"]
  removeOrphans: false

source:
  - name: vcenter67.corp
    type: vmware
    hostname: vcenter67.corp
    username: [email protected]
    password: "password"
    clusterSiteRelations:
      - .* = Foo
    hostSiteRelations:
      - .* = Foo
    clusterTenantRelations:
      - .* = ACME
    hostTenantRelations:
      - .* = ACME
    vmTenantRelations:
      - .* = ACME

What is the problem?

VLAN is not created in Netbox.

Steps to reproduce

run netbox-ssot -config ./config.yml

What did you expect to happen?

It should work

What actually happened?

nicolas@MacBook-Pro-de-Nicolas:~/go/src/github.com/n-rodriguez/netbox-ssot$ ./dist/dynamic_darwin_arm64_v8.0/netbox-ssot -config ./config.yml
Running version unknown built on unknown (commit unknown)

Netbox-SSOT has started at 2024-11-28T01:53:59+01:00
2024/11/28 01:54:13 ERROR   (main): unexpected status code: 400: {"__all__":["VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo."]}: initVlans

Share DEBUG-level logs (remove sensitive information)

2024/11/28 02:46:36 add_items.go:714    DEBUG   (inventory): Vlan VM Network does not exist in Netbox. Creating it...
2024/11/28 02:46:36 rest.go:137         DEBUG   (inventory): Creating objects.Vlan with path /api/ipam/vlans/ with data: Vlan{ID: 96, Name: VM Network, Vid: 10, Status: active}
2024/11/28 02:46:36 main.go:69          ERROR   (main): unexpected status code: 400: {"__all__":["VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo."]}: initVlans

Share your environment details

No response

The text was updated successfully, but these errors were encountered:

n-rodriguez · 2024-11-28T01:56:03Z

Actually vlan should be scoped to site if site exists.

Example :

Vlan{ID: 90, Name: VM Network, Vid: 10, Status: active, Site: 13}
Vlan{ID: 96, Name: VM Network, Vid: 10, Status: active, Site: 14}

From netbox-sync : https://github.com/bb-Ricardo/netbox-sync/blob/main/module/sources/vmware/connection.py#L1868

n-rodriguez · 2024-11-28T02:01:56Z

@bb-Ricardo can you please confirm this is the current implementation in netbox-sync ? Thank you!

n-rodriguez · 2024-11-28T02:24:28Z

It seems to come from here https://github.com/bl4ko/netbox-ssot/blob/main/internal/netbox/inventory/add_items.go#L695

IMHO AddVlan should be improved to take Site or Site.ID in consideration.

bl4ko · 2024-11-28T07:48:55Z

Oh, I see this is a limitation on the netbox's side. If a vlan is assigned to vlangroup, and vlangroup is not assigned to a site, this vlan can't be assigned a site, because site isn't the same as vlangroup's one.

So I guess to implement this feature, I need to change the logic of assigning vlangroup to vlans. If vlan is a assigned a site than the vlangroup for this vlan should also be part of the site...

bb-Ricardo · 2024-11-28T07:55:20Z

Hi,

indeed, the Interface, IP and VLAN syncing is not trivial for vCenter objects:

The logic is implemented here: https://github.com/bb-Ricardo/netbox-sync/blob/3c07a4c001443f50d95ae4668c931599e41efeae/module/sources/common/source_base.py#L234

this function is used on each interface of a Host or VM:

extract object site name and tenant
iterate over each discovered interface IP addresses
- try to find prefix for IP in NetBox. site specific first, fall back to global prefix
- if no prefix can be found, IP address can't be synced to NetBox
- sanity check if defined prefix length for IP matches prefix length configured in NetBox
- extract VRF data and tenant data from prefix if present
- add prefix length to IP address if IP address was discovered without an prefix length
- try to find the IP address object in NetBox
- check if the possible prefix VRF matches the configured IP VRF (need to be the same)
- check if the IP has been reassigned to different interface and/or device
  - check if current interface status is active or not
  - check if the discovery interface status is active or not
    - move ip from disabled to enabled interface
    - postpone reassignment of IP address if both interfaces are (at this state of sync) are enabled
- based on config assign Host/VM tenant to IP address
- assign discovered VRF to IP address
- remove IP addresses/interface association if IP address is no longer assigned to any interface
iterate over all matching IP prefixes and try to find:
- assigned VLAN from IP prefix for untagged interfaces
- assigned VLANs from IP prefixes for tagged interfaces
iterate over all tagged and untagged VLANs discovered for the interface
- try to find a matching site specific VLAN if prefix did not have VLAN assigned
- try to find a matching global VLAN if prefix did not have VLAN assigned
- check if VLAN should even be synced according to config
- assign VLAN group according to config

This has been done to comply to the object relations and constrains within NetBox

bb-Ricardo · 2024-11-28T07:59:24Z

Oh, I see this is a limitation on the netbox's side. If a vlan is assigned to vlangroup, and vlangroup is not assigned to a site, this vlan can't be assigned a site, because site isn't the same as vlangroup's one.

So I guess to implement this feature, I need to change the logic of assigning vlangroup to vlans. If vlan is a assigned a site than the vlangroup for this vlan should also be part of the site...

Yes, the VLAN Group needs to be defined with the site constrain and matched accordingly.

To give the user the flexibility to configure this, these config options have been added:
https://github.com/bb-Ricardo/netbox-sync/blob/13685c5b6018ba032f770c0af9a8264985b16833/settings-example.ini#L326-L344

bb-Ricardo · 2024-11-28T08:03:51Z

And I forgot:

After all sources are synced, then the IP address assignment for active and non active interfaces are reevaluated and set accordingly.

Only after you have discovered all objects and interfaces you are able to tell for sure if an IP address has been moved from one object/interface to another. And if this was the primary IP then this primary IP has to be first removed from the object, then the IP needs to be assigned to the new object interface and then the primary can be defined to the newly assigned object.

Otherwise NetBox will complain all the time.

bl4ko · 2024-11-28T08:15:45Z

Hi, thanks for all of the resources. I see this is a quite complex problem. And this logic must not only be implemented for vcenter, but also for other sources, so I will try to create common functions that can be shared. I will dive into it.

bb-Ricardo · 2024-11-28T08:52:46Z

Hi, thanks for all of the resources. I see this is a quite complex problem. And this logic must not only be implemented for vcenter, but also for other sources, so I will try to create common functions that can be shared. I will dive into it.

I know, this took a lot of iterations to get to this state (still believe it's missing some edge cases). I also moved this to the common source section as it is indeed, a common concept of assigning relations.

n-rodriguez · 2025-01-09T18:21:39Z

@bl4ko Hi there! Any news? Let me know if you need something to get this done 😏 Thank you!

bl4ko · 2025-01-15T08:52:05Z

I have added option to tackle this problem in the following way:

    vlanGroupRelations:
      - .* = FooVlanGroup
    vlanSiteRelations:
      - .* = Foo
    vlanGroupSiteRelations:
      - .* = Foo

n-rodriguez · 2025-01-15T15:54:56Z

@bl4ko thanks for working on this! your solution seems reasonable but I still have issues during synchronization :

2025/01/15 16:03:19 add_items.go:714    DEBUG   (inventory): Vlan VM Network does not exist in Netbox. Creating it...
2025/01/15 16:03:19 rest.go:137         DEBUG   (inventory): Creating objects.Vlan with path /api/ipam/vlans/ with data: Vlan{ID: 96, Name: VM Network, Vid: 10, Status: active, Site: Site{Name: Bar}}
2025/01/15 16:03:19 main.go:69          ERROR   (main): unexpected status code: 400: {"__all__":["VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Bar."]}: initVlans

Vlans are wrongly scoped to Group instead of Site : https://github.com/n-rodriguez/netbox-ssot/blob/aff2cc6eecf112bc907e75bb4ce41456d1de6e27/internal/netbox/inventory/init_items.go#L584-L592 and thus netbox-ssot tries to recreate an existing vlan.

bl4ko · 2025-01-16T07:57:56Z

Oh, yeah I see...

bl4ko · 2025-01-16T09:46:05Z

I have pushed another commit, i think this should work now...

n-rodriguez · 2025-01-17T00:42:26Z

I have pushed another commit, i think this should work now...

@bl4ko I've just tested your patch and it still doesn't work 😢

2025/01/17 01:21:35 add_items.go:714    DEBUG   (inventory): Vlan VM Network does not exist in Netbox. Creating it...
2025/01/17 01:21:35 rest.go:137         DEBUG   (inventory): Creating objects.Vlan with path /api/ipam/vlans/ with data: Vlan{ID: 96, Name: VM Network, Vid: 10, Status: active}
2025/01/17 01:21:36 main.go:69          ERROR   (main): unexpected status code: 400: {"__all__":["VLAN is assigned to group BarDefaultVlanGroup (scope: None); cannot also assign to site Bar."]}: initVlans

"VLAN is assigned to group BarDefaultVlanGroup (scope: None); cannot also assign to site Bar."

It seems that netbox-ssot tries to assign both Group and Site which is not possible.

A VLAN either belongs to a Group or a Site but not both.

In our case we only use Site scope. Thank you!

bl4ko · 2025-01-17T07:44:52Z

Ok, thanks for the report. It is possible that both VlanGroup and Site are assigned if VlanGroup is scoped to Site. I will fix this again, since I missed it here:

netbox-ssot/internal/netbox/inventory/helpers.go

Line 21 in a7e8dc0

vlanGroup, err := nbi.AddVlanGroup(ctx, &objects.VlanGroup{

bl4ko · 2025-01-17T09:02:55Z

I have added another commit. Now I am pretty confident this should work...

@n-rodriguez does it work now?

n-rodriguez · 2025-01-18T16:43:49Z

I have added another commit. Now I am pretty confident this should work...

The vlan part has worked but I had another issue with the vmware cluster part. Let me retry on monday. Thank you!

bl4ko self-assigned this Nov 28, 2024

bl4ko added the bug Something isn't working label Nov 28, 2024

bl4ko added a commit that referenced this issue Jan 15, 2025

fix: adding site-vlan-group-relation (#358)

cc7cde8

bl4ko added a commit that referenced this issue Jan 15, 2025

fix: partial fix for #358

4c744e3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo. #358

[Bug] VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo. #358

n-rodriguez commented Nov 28, 2024 •

edited

Loading

n-rodriguez commented Nov 28, 2024 •

edited

Loading

n-rodriguez commented Nov 28, 2024 •

edited

Loading

n-rodriguez commented Nov 28, 2024

bl4ko commented Nov 28, 2024

bb-Ricardo commented Nov 28, 2024

bb-Ricardo commented Nov 28, 2024

bb-Ricardo commented Nov 28, 2024 •

edited

Loading

bl4ko commented Nov 28, 2024 •

edited

Loading

bb-Ricardo commented Nov 28, 2024

n-rodriguez commented Jan 9, 2025

bl4ko commented Jan 15, 2025

n-rodriguez commented Jan 15, 2025

bl4ko commented Jan 16, 2025

bl4ko commented Jan 16, 2025 •

edited

Loading

n-rodriguez commented Jan 17, 2025

bl4ko commented Jan 17, 2025

bl4ko commented Jan 17, 2025 •

edited

Loading

n-rodriguez commented Jan 18, 2025

[Bug] VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo. #358

[Bug] VLAN is assigned to group Default netbox-ssot vlan group (scope: None); cannot also assign to site Foo. #358

Comments

n-rodriguez commented Nov 28, 2024 • edited Loading

Is this urgent?

How are you running netbox-ssot?

What is the version of netbox-ssot

Which module has the issue?

Share your configuration

What is the problem?

Steps to reproduce

What did you expect to happen?

What actually happened?

Share DEBUG-level logs (remove sensitive information)

Share your environment details

n-rodriguez commented Nov 28, 2024 • edited Loading

n-rodriguez commented Nov 28, 2024 • edited Loading

n-rodriguez commented Nov 28, 2024

bl4ko commented Nov 28, 2024

bb-Ricardo commented Nov 28, 2024

bb-Ricardo commented Nov 28, 2024

bb-Ricardo commented Nov 28, 2024 • edited Loading

bl4ko commented Nov 28, 2024 • edited Loading

bb-Ricardo commented Nov 28, 2024

n-rodriguez commented Jan 9, 2025

bl4ko commented Jan 15, 2025

n-rodriguez commented Jan 15, 2025

bl4ko commented Jan 16, 2025

bl4ko commented Jan 16, 2025 • edited Loading

n-rodriguez commented Jan 17, 2025

bl4ko commented Jan 17, 2025

bl4ko commented Jan 17, 2025 • edited Loading

n-rodriguez commented Jan 18, 2025

n-rodriguez commented Nov 28, 2024 •

edited

Loading

n-rodriguez commented Nov 28, 2024 •

edited

Loading

n-rodriguez commented Nov 28, 2024 •

edited

Loading

bb-Ricardo commented Nov 28, 2024 •

edited

Loading

bl4ko commented Nov 28, 2024 •

edited

Loading

bl4ko commented Jan 16, 2025 •

edited

Loading

bl4ko commented Jan 17, 2025 •

edited

Loading