Decouple stream bypass from TLS encrypted bypass v7

[lukashino]

Following up on #12082

Redmine ticket: https://redmine.openinfosecfoundation.org/issues/6788

Describe changes:
v7

• Style guide changes as suggested in the prev PR
• Encryption Handling has now three states, similar to TLS
• rebased

v6

• rebased

v5

• rebased
• added upgrade section
• fixed docs - Thanks Juliana
• SV tests should pass now

v4

• rebased
• changed SSH bypass defaults to hopefully be in sync with the previous settings

v3

• added SSH app-layer option encryption-handling allowing to choose whether to continue inspection on SSH once it turns encrypted
• added SV tests
• minor docs updates

SV_BRANCH=OISF/suricata-verify#2171

[github-actions]

NOTE: This PR may contain new authors.

[codecov]

Codecov Report

Attention: Patch coverage is 84.21053% with 6 lines in your changes missing coverage. Please review.

Project coverage is 83.21%. Comparing base (a9b36d8) to head (d4aa5a3).

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #12249      +/-   ##
==========================================
+ Coverage   83.18%   83.21%   +0.03%     
==========================================
  Files         912      912              
  Lines      257174   257203      +29     
==========================================
+ Hits       213930   214032     +102     
+ Misses      43244    43171      -73     

Flag	Coverage Δ
fuzzcorpus	61.04% <42.10%> (+0.03%)	⬆️
livemode	19.41% <13.15%> (-0.01%)	⬇️
pcap	44.35% <42.10%> (-0.05%)	⬇️
suricata-verify	62.80% <84.21%> (+0.01%)	⬆️
unittests	59.18% <42.10%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[suricata-qa]

Information:

ERROR: QA failed on SURI_TLPR1_alerts_cmp.

field	baseline	test	%
	SURI_TLPR1_stats_chk
.flow.end.state.local_bypassed	26013	17669	67.92%
.app_layer.flow.dcerpc_tcp	40	42	105.0%
	IPS_AFP_stats_chk
.flow.end.state.local_bypassed	1080	0	-
.flow_bypassed.local_pkts	25920	0	-
.flow_bypassed.local_bytes	2833920	0	-

Pipeline 23726

[lukashino]

Re QA results:
Expected behavior change, the QA needs to set ssh encryption handling to bypass to reproduce the same results.

[victorjulien]

Re QA results: Expected behavior change, the QA needs to set ssh encryption handling to bypass to reproduce the same results.

This suggests a behavior change in default config, which will likely get some ppl caught off guard on upgrades. How should that be handled?

[lukashino]

The default behavior is not to bypass SSH after the encrypted part so that should be the same.
The default behavior is now the same as TLS so after encrypted traffic starts the flow is tracked but not inspected. Previously the traffic would be fully inspected even with encrypted traffic if stream.bypass would be false.

Upgrade guide contains a mention about this behavior change and it contains info how to unify the bypass behavior if stream.bypass + tls.encrypted bypass was set.

[victorjulien]

So previously having stream.bypass would lead to ssh encrypted traffic bypass, but after this PR the exact same config won't bypass the ssh encrypted traffic, right? So this is something that will affect anyone who upgrades with an existing yaml. This is what the Qalab result shows as well. One way to address this would be to simply default the new ssh option to bypass, I think?

[catenacyber]

I agree that SSH should keep its default behavior of getting bypassed

[lukashino]

One way to address this would be to simply default the new ssh option to bypass

That will reproduce the behavior for people with stream.bypass=true but it will bypass SSH connections for people without it. And the default suricata.yaml.in setting is with stream.bypass disabled.

[lukashino]

So, rather, maybe we can change the default behavior of stream.bypass to true and we can also set TLS and SSH encryption handling to "bypass". I think usually people want the traffic to be bypassed, so it will behave the same. For people, who want to inspect all the traffic - they will need to reconfigure the SSH/TLS behavior to "default" or "full" together with stream.bypass setting.

[lukashino]

The behavior change here is also that if stream.bypass == false then TLS and SSH will stop inspecting the traffic (but keep tracking- "default") after the encrypted part. This would not happen previously because the "default" option was conditioned with stream.bypass set to true.