RFC: Firewall use case

[njlavigne]

An "RFC" with a collection of use cases, feature ideas, and opportunities related to using Suricata in a firewall role.

I don't expect to merge this, but look forward to the discussion.

[victorjulien]

First of all, thanks a lot for this write up Jamie. It's great. Side note: we've been debating RFCs as a tool for a while, and I think this a great example of how it can be used (or at least started :) )

Some incomplete initial thoughts:

I agree a different type of rule-set should be made possible. In this new set type we should use a different approach: explicit everything, no automagic behind the scenes changing behavior (e.g. IP-only), no complex and hard to follow conditions leading to vastly different outcomes (e.g. pass pkt vs pass flow).

I think our first step should be to define a scheme for this, then worry about how to get there is a reasonable backwards compatible way later.

I've often thought about allowing users to annotate rule files with specific config directives, but rule managers (including suricata-update) generally create a single rule file out of the many, so it may not be helpful to mark individual rule files as different types. So instead, perhaps Suricata can just load files differently.

E.g. something explicit like

default-rule-path: /var/lib/suricata/rules
rule-files:
  - suricata.rules
firewall-rule-path: /var/lib/suricata/fwrules
firewall-rule-files:
  - firewall.rules

This could then impose a strict set of conditions on the rules in that file, as well as the individual rules:

• the rules in the file would never change ordering.
• rules would be force to only pick explicit detection hooks and explicit actions
  e.g. drop:pkt tcp:pre-stream any any -> any any (tcp.flags:R+; dsize:>0; msg:"DROP RST with data before the stream engine";)
• Explicit actions would require clear documentation obviously
• Hooks need to be clearly defined
• Optimizations can never change the behavior.

An example of something we do today, but should not do in this idea, is the current IP-only logic. Currently rules are evaluated after parsing to see if they meet the "IP-only" conditions. If so, they automagically change their behavior in significant ways:

1. inspection happens only for first packet per flow dir
2. actions apply to the flow
3. they are evaluated before other rules, so violating any ordering logic

In my pseudo rule lang example above, we could get to the same result using something like:

pass:flow ip:flow_start_bidir 1.2.3.4 any -> any any ();

The engine could then still optimize it behind the scenes to use a radix tree or similar, as long as the other conditions are not violated.

I guess a fundamental decision that this would require is: how would these rules interact with "common" IDS/IPS rules?

Here an analogy with a common IPS setup may be helpful.

When building a firewall on linux, a common setup is:

1. an (nf|ip)tables rule-set for managing an IP level default-DENY policy
2. for any traffic that would be ACCEPTed, instead use NFQUEUE
3. Suricata will thus only receive (over NFQUEUE) the traffic within the constraints of the iptables rule-set, and can apply it's IPS signatures to drop the bad traffic and pass the rest.

I wonder if that is useful as a goal to replicate. Like a 2 stage process.

1. enforce rule based policy (based on IP/TCP but also app-layer things like SNI)
2. inspect what matches the policy in (1) against the more threat detection oriented signature set

I guess this would be expressed more like

policy-rule-path: /var/lib/suricata/policy-rules
policy-rule-files:
  - policy.rules # e.g. bob can talk to alice
signature-rule-path: /var/lib/suricata/signatures
signatures-rule-files:
  - signatures.rules # e.g. ET open rules-set for detecting various attacks

Rules vs signatures?

In the suricata docs, code and general terminology we use "signatures" and "rules" interchangeably. However this feels like a mistake. The current engine is very much a signature engine, where we're looking for the needle in a haystack and we can reorder and interpret things to a large extend to do so in an optimized way. The firewall use-case wants to see a policy engine, with clear, explicit and predictable rules.

I don't know if it is too late for us, but we can try to separate these terms to reflect these different meanings. Alternatively, we need something new. E.g. a "policy rule" or something.

[njlavigne]

First of all, thanks a lot for this write up Jamie. It's great. Side note: we've been debating RFCs as a tool for a while, and I think this a great example of how it can be used (or at least started :) )

Happy to help contribute to this - it was Juliana that suggested an RFC format when I came looking for an approach that would work for this kind of brainstorming.

I've often thought about allowing users to annotate rule files with specific config directives, but rule managers (including suricata-update) generally create a single rule file out of the many, so it may not be helpful to mark individual rule files as different types. So instead, perhaps Suricata can just load files differently.

I like the idea of a separate rule or file type. You didn't say it directly but if Suricata were to introduce a new "firewall" rule type then I think it's important that these rules cannot be confused for IDS rules - so an IDS rule could not be loaded as a firewall rule and vice versa. If the firewall rules were to require the explicit actions like drop:pkt that would be enough to distinguish them and allow the rest of the structure to remain familiar enough that they would make sense to users.

All of my concerns around backward compatibility in this case would only apply to the new firewall policy rules, and the IDS part of the engine can retain the freedom to change interpretations of IDS signatures where it makes sense to do so.

I wonder if that is useful as a goal to replicate. Like a 2 stage process.

1. enforce rule based policy (based on IP/TCP but also app-layer things like SNI)
2. inspect what matches the policy in (1) against the more threat detection oriented signature set

Yes, absolutely - I didn't mention it here but I have separately been looking for a way to do this. Separating the two rulesets is ideal for this because I think a very common case will be for users to author their own firewall policy, while continuing to use a set of threat detection signatures sourced from an external provider.

I don't know if it is too late for us, but we can try to separate these terms to reflect these different meanings. Alternatively, we need something new. E.g. a "policy rule" or something.

From my experience I would expect that updating docs/code would be doable, but getting a large existing userbase to adopt the change consistently will be much harder. So if it's one or the other then introducing some new terminology for a new feature would certainly be easier. Unfortunately in the firewall world "rules" are an already well-established term too - "firewall rules" could be an option but there will be a tendency to shorten it to just "rules", which is ambiguous.

However if the rule types are distinct and not interchangeable (as above) then this may not be a real problem in practice? You could say "firewall rule" or "IDS rule" when it's necessary to disambiguate but otherwise it should be clear from the context.

[jasonish]

In the suricata docs, code and general terminology we use "signatures" and "rules" interchangeably. However this feels like a mistake. The current engine is very much a signature engine, where we're looking for the needle in a haystack and we can reorder and interpret things to a large extend to do so in an optimized way. The firewall use-case wants to see a policy engine, with clear, explicit and predictable rules.

I mostly agree, but the naming could still be confusing? How much of a signature could a firewall rule contain? You mention TLS SNI, would we maybe restrict what could go in these rules?

[njlavigne]

I mostly agree, but the naming could still be confusing? How much of a signature could a firewall rule contain? You mention TLS SNI, would we maybe restrict what could go in these rules?

I think that would be the way I would approach it - starting with a restricted smaller set of supported features such that the behavior could be very well specified & tested, and expand it over time.

Would it help if I were to provide a list of what I see as the important core features as a starting point?

[victorjulien]

I mostly agree, but the naming could still be confusing? How much of a signature could a firewall rule contain? You mention TLS SNI, would we maybe restrict what could go in these rules?

I think that would be the way I would approach it - starting with a restricted smaller set of supported features such that the behavior could be very well specified & tested, and expand it over time.

Would it help if I were to provide a list of what I see as the important core features as a starting point?

Yes absolutely!

[jufajardini]

Really appreciate the effort put into sharing your experience and suggestions here, Jamie. It helps me have a better view of many things.

As you know, we're working on documenting the rule types - and thanks for your feedback there, btw. The ticket for documenting the --engine-analysis output is something I would also like to tackle soon.

Rules vs signatures?

Agreeing with Jamie here that new terminology could be better.

I wonder if that is useful as a goal to replicate. Like a 2 stage process.

+ 1 on this, especially with the explicit rule language

[jufajardini]

Wonder if it would be helpful to highlight this RFC on discord, listing main points for discussion, to see if we get more input.

[jufajardini]

Good point. Pondering how/ where to document this.

I think the place where we touch this indirectly is when discussing the Exception Policies, but as that's not the goal there, it's not a full description.

Should at least add a ticket to track this.

[jufajardini]

Well, there's one, already https://redmine.openinfosecfoundation.org/issues/5465