Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

detect: log app-layer metadata in alert with single tx v5 #12168

Closed

Conversation

catenacyber
Copy link
Contributor

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/7199

May also solve https://redmine.openinfosecfoundation.org/issues/7406 and https://redmine.openinfosecfoundation.org/issues/7350

Describe changes:

  • detect: log app-layer metadata in alert with single tx

SV_BRANCH=OISF/suricata-verify#2146

#12166 with big changes

  • Uses a suricata.yaml config option detect.force-applayer-findtx
  • output alert with tx_id_forced: true in such cases
  • documentation !
  • removes the check PACKET_ALERT_FLAG_STREAM_MATCH as overridden by the new one : one live tx + config ok

Answering #12166 (comment) :

I would like to see a per protocol analysis of how we can be sure that this condition provides us with a relevant tx.

I think/fear that for every protocol, there does not exist a condition robust enough to provide the relevant tx for all use cases = set of rules + people expectations

Do you have such a condition ?

We've decided it's worse to log the wrong one, than to log none.

I think there is quite a demand to log something, and since I think we cannot be always right, we can ask the user to turn on some configuration parameter to ask for it

Ticket: 7199

Uses a config parameter detect.force-applayer-findtx to enable
this behavior (off by default)

This feature is requested for use cases with signatures not
using app-layer keywords but still targetting application
layer transactions, such as pass/drop rule combination,
or lua usage.

This overrides the previous behavior of checking if the signature
has a content match, by checking if there is only one live
transaction, in addition to the config parameter being set.
Copy link

codecov bot commented Nov 28, 2024

Codecov Report

Attention: Patch coverage is 73.33333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 45.33%. Comparing base (bd7d38e) to head (e5d50ce).
Report is 25 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #12168      +/-   ##
==========================================
- Coverage   49.81%   45.33%   -4.49%     
==========================================
  Files         909      909              
  Lines      257904   257736     -168     
==========================================
- Hits       128467   116832   -11635     
- Misses     129437   140904   +11467     
Flag Coverage Δ
fuzzcorpus 60.97% <73.33%> (+0.02%) ⬆️
livemode 19.43% <33.33%> (+<0.01%) ⬆️
pcap 44.41% <73.33%> (-0.01%) ⬇️
suricata-verify ?
unittests 8.98% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPR1_stats_chk
.http.byterange.memuse 99602115 104195456 104.61%

Pipeline 23591

Copy link
Contributor

@jufajardini jufajardini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do hope we find something we can agree upon with this work :)

Left a couple of comments for the docs/ expressions used part.

Comment on lines +84 to +85
The alert event will have ``"tx_id_forced": true`` to recognize
these alerts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pondering on the name here (as it will be used in the log output).
tx_match_forced? tx_logged_forced?

Copy link
Member

@jasonish jasonish Nov 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match has too much meaning with rule matching in Suricata-language, so I'd avoid that.

etc/schema.json Show resolved Hide resolved
doc/userguide/upgrade.rst Show resolved Hide resolved
@jasonish
Copy link
Member

Are we only logging in the case there is one TX? findtx doesn't match that very well, as we're not finding one. Its a simpler check than that. I fear findtx implies too much we're finding the correct one.

@catenacyber
Copy link
Contributor Author

Are we only logging in the case there is one TX? findtx doesn't match that very well, as we're not finding one. Its a simpler check than that. I fear findtx implies too much we're finding the correct one.

Yes logging if one tx and config parameter set

Please give me better names :-)

# try to find an app-layer transaction for rules without app-layer keywords
# allows to log app-layer metadata in alert
# but the transaction may not be the relevant one.
# force-applayer-findtx: no
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

force-tx-log?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... I am not sure.
This is the detect engine doing the work here, not the logging...

@catenacyber
Copy link
Contributor Author

Next in #12195

@catenacyber catenacyber closed this Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants