Expanded operation support and usage in Nix community

[leifwalsh]

Welcome to the party, @flokli and @ninjatrappeur!

I wanted to start a place where we could discuss things outside the context of a PR or issue.

We're excited to have you here and that you're finding our project promising. :) We're going to be careful about expanding nsncd's complexity, because it's such a low level part of our systems. I filed #42 and #43 as ideas for making it less risky to add functionality, I'm interested to hear what you think of those ideas, and maybe you'd find them useful too.

I think you might find as you get more experience running this that you'll want to explore some of the design space around the workers. We tried a few ideas and currently have something that works for us, but the design space is rich:

• Whether to use threads or processes (processes may have better isolation properties for misbehaving NSS plugins)
• Whether to use blocking semantics or async (I'm not actually sure whether this is meaningful yet)
• Whether to prefork workers
• How to handle backpressure (delay responses, drop them, try to scale up)

I am interested to see if you encounter use cases that might require changing the worker model, and if so, to discuss and learn from your use cases.

We'd definitely like to collaborate with you and work to incorporate your changes in our project. But this is not the main day-to-day job for any of us, so if we're too slow to respond feel free to keep experimenting in your forks and we'll try to keep up with your pace.

A clerical note: we're supposed to have you sign a Contributor License Agreement, but we forgot to prepare one for you to sign! I'm going to get one prepared and add it to the repo along with a CONTRIBUTING.md ASAP, hang tight please.

[flokli]

@leifwalsh can you create separate discussions/issues for the bullet points? It probably makes sense to discuss these separately.

[leifwalsh]

Let's use the current discussion for general communication about how we should collaborate.

We don't currently have plans to change the request handling model, so please consider those bullet points just advice from me to you about what to think about. If you want to discuss any of them in more detail, especially if you have benchmarks that motivate you in some direction, maybe open issues about that? Or discussions. I don't think you should be prevented from creating your own discussion topics, but if you are that's something I didn't intend, let me know and I'll try to change the permissions or something.

[flokli]

@leifwalsh has there been any update on your side with how we can proceed with this?

So far, some more fixes in the host lookups were found (for IPv6-only hosts), and there are some more refactorings queued (nix-community#2). And I heard some people have been itching to do some more refactorings ;-)

We'll be happy to cherry-pick stuff together, and provide a downstream main branch at nix-community/nsncd, but I'd personally still like to see these changes finding its way upstream somehow. Is there a way to unblock outside contributions?

One of the ideas is that this nsncd can be used to fix NSS resolution when running nix-built binaries on non-NixOS systems, but if you need to run our fork of it, it kinda makes things unnecessarily complicated…

[leifwalsh]

Sorry about that, work got busy the last couple weeks.

I've added the CLA we need in #50. If you could fill that out and send it over we can start working to merge your changes in.

[picnoir]

CLA signed and sent.

[flokli]

Same here.

[flokli]

@leifwalsh I published https://flokli.de/posts/2022-11-18-nsncd/, which explains our motivations a bit. With the CLA being signed, it would be nice if we could get this integrated into nsncd upstream.

[leifwalsh]

Thanks! I worked a bit on #42 today, I think we'll feel better about merging your work once we know we can disable it if we have to.

[leifwalsh]

We plan to flip the default for the release after NixOS 22.11.

Am I reading this right, that you're going to make this a default component of NixOS? Or for some subset of users?

[flokli]

As explained in the article, NixOS needs something responding to NSS requests on the other side of the socket to properly work. I'd rather have this be nsncd than nscd, due to the caching issues and instability in the latter.

[flokli]

The default has been switched, NixOS Unstable and all new releases will now use our nsncd fork.

It would be nice if #49 could get merged, which requires #54 (comment) to get unstuck.