Use keep-alive

[DL6ER]

What does this implement/fix?

Use HTTP/1.1 keep-alive to allows clients to reuse TCP connection for subsequent HTTP requests, which can improve performance dramatically as no further TLS handshakes and connection opening/closing are required. This reduces pressure especially on low-end hardware lacking hardware-crypto for TLS.

This is disabled by default in CivetWeb as it puts additional compliance constraints on the request handlers. However, Pi-hole's API already fully complies with these additional constraints so no other changes than enabling this are needed.

We set a default timeout of 5 seconds after which the server closes the ready-to-be-used-again connections to free them for other requests. We intentionally keep such a long timeout so that requests being fired on the background (e.g. the regular update of GET /api/summary) can benefit here as well without needing additional handshakes.

Note that, while this reduces the overhead for opening and closing connections when loading several resources from one server, it also blocks one port and one thread at the server during the lifetime of this connection. Unfortunately, most browsers do not seem to close the keep-alive connection after loading all resources required to show a website. The server closes a keep-alive connection, if there is no additional request from the client during this timeout. If there are really many clients using the webserver at the same time, this may require you to increase webserver.threads when you see timeouts.

Note

Don't get me wrong here. This does not reduce the number of TLS handshakes from, say, 35 to 1. It just allows already established connections to be reused. If the web interface makes five requests exactly parallel, there will be five handshakes.
However, if requests happens sequentially, the total number of handshakes is drastically reduced (see TL;DR below). Even on my fairly fast x86_64 microserver running Pi-hole the speed enhancement was noticeable.

TL;DR: On development, we see roughly 70 individual TLS handshakes but only six on this branch.

Before this PR

Each connection needed a connection-establishment incl. a TLS handshake on its own.

After this PR

Only the first connection needs the full handshake, following connections can reuse the existing connection without having to do any additional handshaking on their own.

---

Related issue or feature (if applicable): N/A

Pull request in docs with documentation (if applicable): N/A

---

By submitting this pull request, I confirm the following:

1. I have read and understood the contributors guide, as well as this entire template. I understand which branch to base my commits and Pull Requests against.
2. I have commented my proposed changes within the code.
3. I am willing to help maintain this change if there are issues with it later.
4. It is compatible with the EUPL 1.2 license
5. I have squashed any insignificant commits. (git rebase)

Checklist:

• The code change is tested and works locally.
• I based my code and PRs against the repositories developmental branch.
• I signed off all commits. Pi-hole enforces the DCO for all contributions
• I signed all my commits. Pi-hole requires signatures to verify authorship
• I have read the above and my PR is ready for review.

[PromoFaux]

I played with this option when I was trying to hunt down slowness issues after initial release - I never got around to investigating it fully, though. LGTM

[PromoFaux]

I played with this option when I was trying to hunt down slowness issues after initial release - I never got around to investigating it fully, though. LGTM

[XhmikosR]

Confirmed! I was right to think something was missing :)