Batch backports to main-7.0.x

doc/userguide/configuration/exception-policies.rst

@@ -7,7 +7,10 @@ Suricata has a set of configuration variables to indicate what should the engine
 do when certain exception conditions, such as hitting a memcap, are reached.
 
 They are called Exception Policies and are configurable via suricata.yaml. If
-enabled, the engine will call them when it reaches exception states.
+enabled, the engine will call them when it reaches exception states. Stats for
+any applied exception policies can be found in counters related to the specific
+configuration setting (:ref:`read more<eps_stats>`). Some configuration is
+available directly via the :ref:`stats settings<suricata_yaml_outputs>`.
 
 For developers or for researching purposes, there are also simulation options
 exposed in debug mode and passed via command-line. These exist to force or
@@ -62,33 +65,40 @@ Specific settings
 Exception policies are implemented for:
 
 .. list-table:: Exception Policy configuration variables
-   :widths: 20, 18, 62
+   :widths: 18, 18, 18, 44
    :header-rows: 1
 
    * - Config setting
      - Policy variable
+     - Affects
      - Expected behavior
    * - stream.memcap
      - memcap-policy
+     - Flow or packet
      - If a stream memcap limit is reached, apply the memcap policy to the packet and/or
        flow.
    * - stream.midstream
      - midstream-policy
+     - Flow
      - If a session is picked up midstream, apply the midstream policy to the flow.
    * - stream.reassembly.memcap
      - memcap-policy
+     - Flow or packet
      - If stream reassembly reaches memcap limit, apply memcap policy to the
        packet and/or flow.
    * - flow.memcap
      - memcap-policy
+     - Packet
      - Apply policy when the memcap limit for flows is reached and no flow could
        be freed up. **Policy can only be applied to the packet.**
    * - defrag.memcap
      - memcap-policy
+     - Packet
      - Apply policy when the memcap limit for defrag is reached and no tracker
        could be picked up. **Policy can only be applied to the packet.**
    * - app-layer
      - error-policy
+     - Flow or packet
      - Apply policy if a parser reaches an error state. Policy can be applied to packet and/or flow.
 
 To change any of these, go to the specific section in the suricata.yaml file
@@ -200,6 +210,43 @@ Notes:
    * Not valid means that Suricata will error out and won't start.
    * ``REJECT`` will make Suricata send a Reset-packet unreach error to the sender of the matching packet.
 
+.. _eps_stats:
+
+Available Stats
+---------------
+
+There are stats counters for each supported exception policy scenario:
+
+.. list-table:: **Exception Policy Stats Counters**
+   :widths: 50 50
+   :header-rows: 1
+   :stub-columns: 1
+
+   * - Setting
+     - Counter
+   * - stream.memcap
+     - tcp.ssn_memcap_exception_policy
+   * - stream.reassembly.memcap
+     - tcp.reassembly_memcap_exception_policy
+   * - stream.midstream
+     - tcp.midstream_exception_policy
+   * - defrag.memcap
+     - defrag.memcap_exception_policy
+   * - flow.memcap
+     - flow.memcap_exception_policy
+   * - app-layer.error
+     - app_layer.error.exception_policy
+
+If a given exception policy does not apply for a setting, no related counter
+is logged.
+
+Stats for application layer errors are available in summarized form or per
+application layer protocol. As the latter is extremely verbose, by default
+Suricata logs only the summary. If any further investigation is needed, it
+is recommended to enable per-app-proto exception policy error counters
+temporarily (for :ref:`stats configuration<suricata_yaml_outputs>`).
+
+
 Command-line Options for Simulating Exceptions
 ----------------------------------------------
 

doc/userguide/configuration/suricata-yaml.rst

@@ -317,6 +317,11 @@ the global config is documented.
       #decoder-events-prefix: "decoder.event"
       # Add stream events as stats.
       #stream-events: false
+      # Exception policy stats counters options
+      # (Note: if exception policy: ignore, counters are not logged)
+      exception-policy:
+        #per-app-proto-errors: false  # default: false. True will log errors for
+                                        # each app-proto. Warning: VERY verbose
 
 Statistics can be `enabled` or disabled here.
 
@@ -339,6 +344,10 @@ Similar to the `decoder-events` option, the `stream-events` option controls
 whether the stream-events are added as counters as well. This is disabled by
 default.
 
+If any exception policy is enabled, stats counters are logged. To control
+verbosity for application layer protocol errors, leave `per-app-proto-errors`
+as false.
+
 Outputs
 ~~~~~~~
 

etc/schema.json

@@ -3750,6 +3750,11 @@
                         "error": {
                             "type": "object",
                             "properties": {
+                                "exception_policy": {
+                                    "description":
+                                            "Consolidated stats on how many times app-layer error exception policy was applied, and which one",
+                                    "$ref": "#/$defs/exceptionPolicy"
+                                },
                                 "bittorrent-dht": {
                                     "$ref": "#/$defs/stats_applayer_error"
                                 },
@@ -4815,6 +4820,11 @@
                         "max_frag_hits": {
                             "type": "integer"
                         },
+                        "memcap_exception_policy": {
+                            "description":
+                                    "How many times defrag memcap exception policy was applied, and which one",
+                            "$ref": "#/$defs/exceptionPolicy"
+                        },
                         "ipv4": {
                             "type": "object",
                             "properties": {
@@ -4951,6 +4961,11 @@
                         "memcap": {
                             "type": "integer"
                         },
+                        "memcap_exception_policy": {
+                            "description":
+                                    "How many times flow memcap exception policy was applied, and which one",
+                            "$ref": "#/$defs/exceptionPolicy"
+                        },
                         "memuse": {
                             "type": "integer"
                         },
@@ -5242,6 +5257,11 @@
                         "midstream_pickups": {
                             "type": "integer"
                         },
+                        "midstream_exception_policy": {
+                            "description":
+                                    "How many times midstream exception policy was applied, and which one",
+                            "$ref": "#/$defs/exceptionPolicy"
+                        },
                         "no_flow": {
                             "type": "integer"
                         },
@@ -5260,6 +5280,11 @@
                         "pseudo_failed": {
                             "type": "integer"
                         },
+                        "reassembly_exception_policy": {
+                            "description":
+                                    "How many times reassembly memcap exception policy was applied, and which one",
+                            "$ref": "#/$defs/exceptionPolicy"
+                        },
                         "reassembly_gap": {
                             "type": "integer"
                         },
@@ -5290,6 +5315,11 @@
                         "ssn_memcap_drop": {
                             "type": "integer"
                         },
+                        "ssn_memcap_exception_policy": {
+                            "description":
+                                    "How many times session memcap exception policy was applied, and which one",
+                            "$ref": "#/$defs/exceptionPolicy"
+                        },
                         "stream_depth_reached": {
                             "type": "integer"
                         },
@@ -5539,6 +5569,11 @@
                 },
                 "internal": {
                     "type": "integer"
+                },
+                "exception_policy": {
+                    "description":
+                            "How many times app-layer error exception policy was applied, and which one",
+                    "$ref": "#/$defs/exceptionPolicy"
                 }
             },
             "additionalProperties": false
@@ -5581,6 +5616,35 @@
                     ]
                 }
             }
+        },
+        "exceptionPolicy": {
+            "type": "object",
+            "properties": {
+                "drop_flow": {
+                    "type": "integer",
+                    "minimum": 0
+                },
+                "drop_packet": {
+                    "type": "integer",
+                    "minimum": 0
+                },
+                "pass_flow": {
+                    "type": "integer",
+                    "minimum": 0
+                },
+                "pass_packet": {
+                    "type": "integer",
+                    "minimum": 0
+                },
+                "bypass": {
+                    "type": "integer",
+                    "minimum": 0
+                },
+                "reject": {
+                    "type": "integer",
+                    "minimum": 0
+                }
+            }
         }
     }
 }

src/Makefile.am

@@ -542,6 +542,7 @@ noinst_HEADERS = \
 	util-enum.h \
 	util-error.h \
 	util-exception-policy.h \
+	util-exception-policy-types.h \
 	util-file-decompression.h \
 	util-file.h \
 	util-file-swf-decompression.h \