Detect rule hook/v10

[victorjulien]

SV_BRANCH=OISF/suricata-verify#2327

This implements part of the idea described here #12167 (comment)

It essentially splits the firewall rules in 2 classes:

1. packet / flow level rules
2. app-layer / tx level rules

Both can be considered to be separate tables or chains of sorts, both with a default drop policy.

So the first step in a ruleset is to allow traffic on a packet level. As an example, lets create a TLS SNI ruleset. We start by allowing the TCP session setup:

pass:packet tcp:all $HOME_NET any <> $EXTERNAL_NET 443 (flow:not_established; alert; sid:1021;)

This is a per packet rule that allows the packets as long as we're not established. In TCP this means the SYN, SYN/ACK, ACK sequence is allowed here.

Then we get to the established state, and now we want the application level rules to take over. To do so, we just a special variant of a pass rule to sort of jump to the application rules:

appfw:packet tcp:all $HOME_NET any <> $EXTERNAL_NET 443 (flow:established; alert; sid:1023;)

The appfw (name TBD) action signals that we consider the packets a pass for the packet ruleset, and jump to the app rules.

Now that we're thinking about tls, lets first make sure we allow the SNI we want:

pass:flow tls:client_hello_done $HOME_NET any -> $EXTERNAL_NET any (tls.sni; content:"www.google.com"; sid:104; alert;)

This will accept the rest of the flow (pass:flow) when the SNI matches. It will do it's evaluation of the rule when the TLS parser hits the client_hello_done state.

We're still making packet level pass/drop decisions, so there is a corner case to handle: fragmented client hello's mean we get packets that are established, but TLS isn't at the client_hello_done state yet. So we pass this as well:

pass:packet tls:client_in_progress $HOME_NET any -> $EXTERNAL_NET any (alert; sid:105;)

If the SNI is not a match, or we receive a client hello w/o SNI, these rules won't match and the default drop policy is activated.

So the full ruleset is

# Packet rules

# allow session setup
pass:packet tcp:all $HOME_NET any <> $EXTERNAL_NET 443 (flow:not_established; alert; sid:1021;)

# pass rest of the flow to 
appfw:packet tcp:all $HOME_NET any <> $EXTERNAL_NET 443 (flow:established; alert; sid:1023;)

# default drop


# App-layer rules

# allow this flow if the SNI is what we expect
pass:flow tls:client_hello_done $HOME_NET any -> $EXTERNAL_NET any (tls.sni; content:"www.google.com"; sid:104; alert;)

# allow tls before client hello is done.
pass:packet tls:client_in_progress $HOME_NET any -> $EXTERNAL_NET any (alert; sid:105;)

# default drop

[codecov]

Codecov Report

Attention: Patch coverage is 86.25107% with 161 lines in your changes missing coverage. Please review.

Project coverage is 80.73%. Comparing base (80dbaac) to head (47e507e).
Report is 32 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #12697      +/-   ##
==========================================
- Coverage   80.75%   80.73%   -0.02%     
==========================================
  Files         934      936       +2     
  Lines      259594   260036     +442     
==========================================
+ Hits       209634   209951     +317     
- Misses      49960    50085     +125     

Flag	Coverage Δ
fuzzcorpus	56.99% <60.58%> (-0.01%)	⬇️
livemode	19.56% <36.04%> (+0.19%)	⬆️
pcap	44.17% <40.83%> (+0.01%)	⬆️
suricata-verify	63.65% <84.14%> (+0.12%)	⬆️
unittests	58.18% <51.92%> (-0.14%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[suricata-qa]

Information: QA ran without warnings.

Pipeline 24943