ja4: adjust and add tests

tests/ja4-quic/README.md

@@ -0,0 +1,119 @@
+# JA4 QUIC
+
+This test checks whether the correct JA4 fingerprints are calculated for the
+given pcap, according to the [reference implementation](https://github.com/FoxIO-LLC/ja4).
+
+## PCAP
+
+Pcap was created on developer machine using a short `tcpdump` session:
+```
+tcpdump -w out.pcap -i wlp61s0 'port 443 and udp'
+```
+
+## Result
+
+`q13d0310h3_55b375c5d22e_cd85d2d88918` which means
+
+* `q`: QUIC
+* `13`: TLS 1.3
+* `d`: SNI is set
+* `03`: 3 cipher suites in Client Hello
+* `10`: 10 extensions in Client Hello
+* `h3`: ALPN protocol
+
+and the hashes of the corresponding sorted extension codes.
+
+
+## Reference output:
+
+```
+$ ../ja4/binaries/linux/ja4 tests/ja4-quic/input.pcap
+- stream: 0
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.201
+  src_port: 51333
+  dst_port: 443
+  tls_server_name: www.blogger.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 1
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.251.209.129
+  src_port: 53371
+  dst_port: 443
+  tls_server_name: socpuppet.blogspot.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 2
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.206
+  src_port: 50440
+  dst_port: 443
+  tls_server_name: apis.google.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 3
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.201
+  src_port: 37252
+  dst_port: 443
+  tls_server_name: www.blogger.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 4
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.206
+  src_port: 57334
+  dst_port: 443
+  tls_server_name: apis.google.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 5
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.185.164
+  src_port: 38677
+  dst_port: 443
+  tls_server_name: www.google.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 6
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.195
+  src_port: 42849
+  dst_port: 443
+  tls_server_name: www.gstatic.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 7
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.251.209.131
+  src_port: 32997
+  dst_port: 443
+  tls_server_name: fonts.gstatic.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 8
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.193
+  src_port: 60461
+  dst_port: 443
+  tls_server_name: 4.bp.blogspot.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 9
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.193
+  src_port: 52446
+  dst_port: 443
+  tls_server_name: 1.bp.blogspot.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+- stream: 10
+  transport: udp
+  src: 192.168.178.25
+  dst: 142.250.181.193
+  src_port: 41171
+  dst_port: 443
+  tls_server_name: 2.bp.blogspot.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+```

tests/ja4-quic/test.yaml

@@ -0,0 +1,14 @@
+requires:
+  min-version: 8.0.0
+  features:
+    - HAVE_JA4
+
+args:
+  - -k none --set app-layer.protocols.tls.ja4-fingerprints=no
+
+checks:
+  - filter:
+      count: 11
+      match:
+        event_type: quic
+        quic.ja4: q13d0310h3_55b375c5d22e_cd85d2d88918

tests/ja4-rules-disabled/README.md

@@ -0,0 +1,13 @@
+# JA4 Disabled Test Case
+
+This test checks -- in cases where JA4 support is disabled -- whether
+using rules that require JA4 support are actually rejected if no "requires"
+keyword is given.
+
+We expect something like:
+
+```
+Error: detect-ja4-hash: JA4 support is not enabled [DetectJa4HashSetup:detect-ja4-hash.c:122]
+Error: detect: error parsing signature "alert quic any any -> any any (msg:"JA4 QUIC Test 1"; ja4.hash; content: "q13d0310h3_55b375c5d22e_cd85d2d88918"; sid:1;)" from file suricata-verify/tests/ja4-rules-disabled/test.rules at line 1 [DetectLoadSigFile:detect-engine-loader.c:183]
+Error: suricata: Loading signatures failed. [LoadSignatures:suricata.c:2427]
+```

tests/ja4-rules-disabled/test.rules

@@ -0,0 +1,2 @@
+alert quic any any -> any any (msg:"JA4 QUIC Test 1"; ja4.hash; content: "q13d0310h3_55b375c5d22e_cd85d2d88918"; sid:1;)
+alert tls any any -> any any (msg:"JA4 TLS Test 1"; ja4.hash; content: "t13d1516h2_8daaf6152771_e5627efa2ab1"; sid:2;)

tests/ja4-rules-disabled/test.yaml

@@ -0,0 +1,9 @@
+requires:
+  min-version: 8.0.0
+  files:
+    - rust/src/ja4.rs
+
+args:
+  - -k none --set app-layer.protocols.tls.ja4-fingerprints=no
+
+exit-code: 1

tests/ja4-rules-invalid/test.rules

@@ -0,0 +1 @@
+alert smb any any -> any any (msg:"JA4 Broken Test 1"; ja4.hash; content: "q"; sid:1;)

tests/ja4-rules-invalid/test.yaml

@@ -0,0 +1,15 @@
+requires:
+  min-version: 8.0.0
+  pcap: false
+  features:
+    - HAVE_JA4
+
+args:
+    - --engine-analysis
+
+exit-code: 1
+
+checks:
+  - shell:
+        args: grep "rule contains conflicting protocols" suricata.log | wc -l | xargs
+        expect: 1

tests/ja4-rules-requires-off/README.md

@@ -0,0 +1,5 @@
+# JA4 Not Compiled In Test Case
+
+This test checks -- in cases where JA4 support is not compiled in -- whether
+using rules that require JA4 support are also skipped if the "requires"
+keyword is given.

tests/ja4-rules-requires-off/test.rules

@@ -0,0 +1,2 @@
+alert quic any any -> any any (msg:"JA4 Requires Test 1"; requires: feature ja4; sid:1;)
+alert tls any any -> any any (msg:"JA4 Requires Test 2"; requires: feature ja4; sid:2;)

tests/ja4-rules-requires-off/test.yaml

@@ -0,0 +1,16 @@
+requires:
+  min-version: 8.0.0
+  script:
+    - ./src/suricata --build-info | grep "JA4 support" | grep no > /dev/null
+
+args:
+  - -k none
+
+checks:
+  - filter:
+      count: 0
+      match:
+        event_type: alert
+  - shell:
+      args: "grep -c 'missing a required feature: feature ja4' stdout"
+      expect: 2

tests/ja4-rules-requires/README.md

@@ -0,0 +1,15 @@
+# JA4 Disabled Test Case -- With `required` Keyword
+
+This test checks -- in cases where JA4 support is disabled -- whether
+using rules that require JA4 support are actually skipped if the "requires"
+keyword is given.
+
+We expect something like:
+
+```
+Info: detect-requires: Suricata did not meet the rule requirements: Suricata missing a required feature: feature ja4 [DetectRequiresSetup:detect-requires.c:38]
+Info: detect: Skipping signature due to missing requirements: alert quic any any -> any any (msg:"JA4 Requires Test 1"; requires: feature ja4; sid:1;) from file /home/satta/tmp/suricata-verify/tests/ja4-rules-requires/test.rules at line 1 [DetectLoadSigFile:detect-engine-loader.c:203]
+Info: detect-requires: Suricata did not meet the rule requirements: Suricata missing a required feature: feature ja4 [DetectRequiresSetup:detect-requires.c:38]
+Info: detect: Skipping signature due to missing requirements: alert tls any any -> any any (msg:"JA4 Requires Test 2"; requires: feature ja4; sid:2;) from file /home/satta/tmp/suricata-verify/tests/ja4-rules-requires/test.rules at line 2 [DetectLoadSigFile:detect-engine-loader.c:203]
+Warning: detect: 1 rule files specified, but no rules were loaded! [SigLoadSignatures:detect-engine-loader.c:359]
+```

tests/ja4-rules-requires/suricata.yaml

@@ -0,0 +1,18 @@
+%YAML 1.1
+---
+
+outputs:
+  - eve-log:
+      enabled: yes
+      filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
+      filename: eve.json
+      types:
+        - alert:
+            payload: no
+            payload-buffer-size: 4kb
+            payload-printable: no
+            packet: no
+            metadata: yes
+        - tls:
+            extended: yes
+        - quic

tests/ja4-rules-requires/test.rules

@@ -0,0 +1,2 @@
+alert quic any any -> any any (msg:"JA4 Requires Test 1"; requires: feature ja4; sid:1;)
+alert tls any any -> any any (msg:"JA4 Requires Test 2"; requires: feature ja4; sid:2;)

tests/ja4-rules-requires/test.yaml

@@ -0,0 +1,16 @@
+requires:
+  min-version: 8.0.0
+  features:
+    - HAVE_JA4
+
+args:
+  - -k none --set app-layer.protocols.tls.ja4-fingerprints=no
+
+checks:
+  - filter:
+      count: 0
+      match:
+        event_type: alert
+  - shell:
+      args: "grep -c 'missing a required feature: feature ja4' stdout"
+      expect: 2

tests/ja4-rules/test.rules

@@ -0,0 +1,4 @@
+alert quic any any -> any any (msg:"JA4 QUIC Test 1"; requires: feature ja4; ja4.hash; content: "q13d0310h3_55b375c5d22e_cd85d2d88918"; sid:1;)
+alert tls any any -> any any (msg:"JA4 TLS Test 1"; requires: feature ja4; ja4.hash; content: "t13d1516h2_8daaf6152771_e5627efa2ab1"; sid:2;)
+alert quic any any -> any any (msg:"JA4 QUIC Test 2"; requires: feature ja4; ja4.hash; content: "q13d0310h3_55b375c5d22e_cd85d2d88918X"; sid:3;)
+alert tls any any -> any any (msg:"JA4 TLS Test 2"; requires: feature ja4; ja4.hash; content: "t13d1516h2_8daaf6152771_e5627efa2ab1X"; sid:4;)

tests/ja4-rules/test.yaml

@@ -0,0 +1,29 @@
+requires:
+  min-version: 8.0.0
+  features:
+    - HAVE_JA4
+
+args:
+  - -k none
+
+checks:
+  - filter:
+      count: 1
+      match:
+        event_type: tls
+        tls.ja4: t13d1516h2_8daaf6152771_e5627efa2ab1
+  - filter:
+      count: 1
+      match:
+        event_type: quic
+        quic.ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+  - filter:
+      count: 1
+      match:
+        event_type: alert
+        alert.signature: JA4 QUIC Test 1
+  - filter:
+      count: 1
+      match:
+        event_type: alert
+        alert.signature: JA4 TLS Test 1

tests/ja4-tls-quic/README.md

@@ -0,0 +1,68 @@
+# JA4 TLS + QUIC
+
+This test checks whether the correct JA4 fingerprints are calculated for the
+given pcap, according to the [reference implementation](https://github.com/FoxIO-LLC/ja4).
+
+## PCAP
+
+Pcap was taken from https://www.cloudshark.org/captures/1737557e3427.
+
+## Result
+
+
+### TCP TLS
+
+`t13d1516h2_8daaf6152771_e5627efa2ab1` which means
+
+* `t`: TCP
+* `13`: TLS 1.2
+* `d`: SNI is set
+* `15`: 15 cipher suites in Client Hello
+* `16`: 16 extensions in Client Hello
+* `h2`: ALPN protocol
+
+and the hashes of the corresponding sorted extension codes.
+
+### QUIC
+
+`q13d0310h3_55b375c5d22e_cd85d2d88918` which means
+
+* `q`: QUIC
+* `13`: TLS 1.2
+* `d`: SNI is set
+* `03`: 3 cipher suites in Client Hello
+* `10`: 10 extensions in Client Hello
+* `h3`: ALPN protocol
+
+and the hashes of the corresponding sorted extension codes.
+
+
+## Reference output:
+
+According to [my issue upstream](https://github.com/FoxIO-LLC/ja4/issues/3):
+
+```
+../ja4/binaries/linux/ja4 tests/ja4-tls-quic/input.pcap
+- stream: 0
+  transport: tcp
+  src: 2001:db8:1::1
+  dst: 2606:4700:10::6816:826
+  src_port: 57098
+  dst_port: 443
+  tls_server_name: cloudflare-quic.com
+  ja4: t13d1516h2_8daaf6152771_e5627efa2ab1
+  ja4l_c: 30_64
+  ja4l_s: 5749_56
+  http:
+  - ja4h: ge20nn16enus_0f5a7a41a252_000000000000_000000000000
+- stream: 0
+  transport: udp
+  src: 2001:db8:1::1
+  dst: 2606:4700:10::6816:826
+  src_port: 50280
+  dst_port: 443
+  tls_server_name: cloudflare-quic.com
+  ja4: q13d0310h3_55b375c5d22e_cd85d2d88918
+  ja4l_c: 113_64
+  ja4l_s: 9285_56
+```

tests/ja4-tls-quic/test.yaml

@@ -0,0 +1,19 @@
+requires:
+  min-version: 7.0.0
+  features:
+    - HAVE_JA4
+
+args:
+  - -k none --set app-layer.protocols.tls.ja4-fingerprints=yes
+
+checks:
+  - filter:
+      count: 1
+      match:
+        event_type: tls
+        tls.ja4: t13d1516h2_8daaf6152771_e5627efa2ab1
+  - filter:
+      count: 1
+      match:
+        event_type: quic
+        quic.ja4: q13d0310h3_55b375c5d22e_cd85d2d88918