Getting as far away from Ansible as possible

[ipspace]

After #2864, we configure all devices the same way:

1. A configuration template is found
2. A device configuration task list (generic or module-specific) is found
3. The device configuration task list is called with the template name to render. It should push the config to the device.

Ansible 12 partially broke template rendering (Jinja2 filters have to use fully-qualified names) and totally borked the device configuration rendering (see #2768). Even worse, it doesn't look like anyone cares anymore. Ansible 13.1 has the exact same problems.

The only sane way forward thus seems to be to get as far away from Ansible as possible:

1. Render all configuration files within netlab (so we'll have predictable outcomes)
2. Use Ansible playbooks solely to push rendered configs to network devices (which might even work with Ansible 12+)
3. Eventually use something else for the "push configuration to device" step. It could be netmiko, nornir or whatever as long as it's relatively stable, Python-based, and consumable as a Python package (so we don't need external dependencies).

(1) and (3) are already implemented for pure containers. Linux, Bird, and dnsmasq are done; activating FRR and VyOS is just a question of a bit of tweaking (bash versus vtysh) and device configuration. I also started a large-scale cleanup of configuration templates, hopefully getting at least (1) done for all supported devices in a reasonable timeframe, which would allow us to implement (2) in time for the 26.01 release (I don't want to prolong this agony any further).

Thoughts/comments? cc @ssasso @jbemmel @DanPartelly @sdargoeuves @Muddyblack @ddutt

[jbemmel]

Scrapli might be a good alternative: https://carlmontanari.github.io/scrapli/

[ipspace]

This might be an issue: https://carlmontanari.github.io/scrapli/user_guide/project_details/#supported-platforms

Looks like Kirk takes a slightly wider approach: https://github.com/ktbyers/netmiko/blob/develop/PLATFORMS.md

[jbemmel]

ok, Kirk wins :)

[DanPartelly]

Yesterday morning, I was enjoying a Makefile maimed with spaces. 50 years since those 6 users cast that TAB shit in stone. An hour later, my distro blessed me with Ansible 13.1.0 . Alegedly, it has a quarter million users, over thousands of companies. OK.

Jokes aside, Netmiko looks good. But if possible, and Nornir's API is simple enough, we might consider using it with the netmiko plugin. We gain inventory, filters, and task abstraction + parallel execution, and if we are clever, we can abstract a bit the plugin backends so next time Voodoo hits, we have options. Nornir's parallelism will shorten config delivery times a lot.

Netmiko is widely packaged. Nornir itself is packaged by a few distros. Nornir-netmiko is not in any repo I know of, but I dumped the transitive closure of their dependencies, and it looks easy to maintain this stack for custom packages.

Using nornir instead of naked netmiko, we introduce more dependencies, but the advantages may be well worth it.

[sdargoeuves]

Nornir seems like a good candidate, I haven't played with it much, so not sure about the task abstraction, but definitely worth it from its inventory/filter/parallelism perspective.

[ipspace]

good news for @DanPartelly: I have working code for #1 and #2 (create configs with netlab, deploy with Ansible), and at least netlab initial works with Ansible 13.1. It's follow-up from #2950, after that one gets merged, I'll submit another series of PRs.

Next step after that: make some containers (Linux, FRR; VyOS) work completely without Ansible.

[ipspace]

Almost there ;) When we add netmiko support, we'll be able to configure most devices without Ansible.

[Muddyblack]

Sounds nice. How is it then handled like it won't have an inventory or such no?

Thinkable to use netlab then also to distribute configuration updates or ios files and updates onto real devices?

[ipspace]

We don't use Ansible inventory within netlab anymore (I hope). Everything we need is in the snapshot pickle. However, netmiko isn't a universal solution (there's NETCONF, gRPC, JSON-RPC...), but it will make Ansible unnecessary in many scenarios.

[Muddyblack]

Sounds great 😁 will be happy to see how it works once its implemented ^

[sdargoeuves]

Nornir is getting some new life: https://opsmill.com/blog/opsmill-is-the-new-steward-of-nornir/

[ipspace]

Nornir is getting some new life: https://opsmill.com/blog/opsmill-is-the-new-steward-of-nornir/

That's really nice to hear, but mostly irrelevant for what we're doing. Nornir is a framework that runs things (sort-of like Ansible Core), the only thing we really need from Ansible are the device configuration modules.