Draft: Security monitoring feature parity with Agent V2

api/grpc/events/v1/security_violation.proto

@@ -0,0 +1,122 @@
+// Copyright (c) F5, Inc.
+//
+// This source code is licensed under the Apache License, Version 2.0 license found in the
+// LICENSE file in the root directory of this source tree.
+syntax = "proto3";
+package events.v1;
+
+option go_package = "events/v1";
+
+// SecurityViolationEvent represents the structured NGINX App Protect security violation data
+message SecurityViolationEvent {
+    // Name of the security policy
+    string policy_name = 1;
+    // Unique support ID for the violation
+    string support_id = 2;
+    // Outcome of the request (e.g., REJECTED, PASSED)
+    string outcome = 3;
+    // Reason for the outcome
+    string outcome_reason = 4;
+    // Reason for blocking exception if applicable
+    string blocking_exception_reason = 5;
+    // HTTP method used
+    string method = 6;
+    // Protocol used (e.g., HTTP/1.1)
+    string protocol = 7;
+    // X-Forwarded-For header value
+    string xff_header_value = 8;
+    // Request URI
+    string uri = 9;
+    // Full request
+    string request = 10;
+    // Indicates if the request was truncated
+    string is_truncated = 11;
+    // Status of the request
+    string request_status = 12;
+    // HTTP response code
+    string response_code = 13;
+    // Server address
+    string server_addr = 14;
+    // Virtual server name
+    string vs_name = 15;
+    // Remote address of the client
+    string remote_addr = 16;
+    // Destination port
+    string destination_port = 17;
+    // Server port
+    string server_port = 18;
+    // List of violations
+    string violations = 19;
+    // List of sub-violations
+    string sub_violations = 20;
+    // Violation rating
+    string violation_rating = 21;
+    // Signature set names
+    string sig_set_names = 22;
+    // Signature CVEs
+    string sig_cves = 23;
+    // Client class
+    string client_class = 24;
+    // Client application
+    string client_application = 25;
+    // Client application version
+    string client_application_version = 26;
+    // Severity of the violation
+    string severity = 27;
+    // Threat campaign names
+    string threat_campaign_names = 28;
+    // Bot anomalies detected
+    string bot_anomalies = 29;
+    // Bot category
+    string bot_category = 30;
+    // Enforced bot anomalies
+    string enforced_bot_anomalies = 31;
+    // Bot signature name
+    string bot_signature_name = 32;
+    // System ID
+    string system_id = 33;
+    // Instance tags
+    string instance_tags = 34;
+    // Instance group
+    string instance_group = 35;
+    // Parent hostname
+    string parent_hostname = 36;
+    // Display name
+    string display_name = 37;
+    // Detailed violation data
+    repeated ViolationData violations_data = 38;
+}
+
+// ViolationData represents individual violation details
+message ViolationData {
+    // Name of the violation
+    string violation_data_name = 1;
+    // Context of the violation
+    string violation_data_context = 2;
+    // Context data associated with the violation
+    ContextData violation_data_context_data = 3;
+    // Signature data for the violation
+    repeated SignatureData violation_data_signatures = 4;
+}
+
+// SignatureData represents signature data contained within each violation
+message SignatureData {
+    // Signature ID
+    string sig_data_id = 1;
+    // Blocking mask
+    string sig_data_blocking_mask = 2;
+    // Buffer information
+    string sig_data_buffer = 3;
+    // Offset in the buffer
+    string sig_data_offset = 4;
+    // Length of the signature match
+    string sig_data_length = 5;
+}
+
+// ContextData represents the context data of the violation
+message ContextData {
+    // Name of the context
+    string context_data_name = 1;
+    // Value of the context
+    string context_data_value = 2;
+}

docs/proto/protos.md

@@ -3,6 +3,12 @@
 
 ## Table of Contents
 
+- [events/v1/security_violation.proto](#events_v1_security_violation-proto)
+    - [ContextData](#events-v1-ContextData)
+    - [SecurityViolationEvent](#events-v1-SecurityViolationEvent)
+    - [SignatureData](#events-v1-SignatureData)
+    - [ViolationData](#events-v1-ViolationData)
+
 - [mpi/v1/common.proto](#mpi_v1_common-proto)
     - [AuthSettings](#mpi-v1-AuthSettings)
     - [CommandResponse](#mpi-v1-CommandResponse)
@@ -95,6 +101,130 @@
 
 
 
+<a name="events_v1_security_violation-proto"></a>
+<p align="right"><a href="#top">Top</a></p>
+
+## events/v1/security_violation.proto
+Copyright (c) F5, Inc.
+
+This source code is licensed under the Apache License, Version 2.0 license found in the
+LICENSE file in the root directory of this source tree.
+
+
+<a name="events-v1-ContextData"></a>
+
+### ContextData
+ContextData represents the context data of the violation
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| context_data_name | [string](#string) |  | Name of the context |
+| context_data_value | [string](#string) |  | Value of the context |
+
+
+
+
+
+
+<a name="events-v1-SecurityViolationEvent"></a>
+
+### SecurityViolationEvent
+SecurityViolationEvent represents the structured NGINX App Protect security violation data
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| policy_name | [string](#string) |  | Name of the security policy |
+| support_id | [string](#string) |  | Unique support ID for the violation |
+| outcome | [string](#string) |  | Outcome of the request (e.g., REJECTED, PASSED) |
+| outcome_reason | [string](#string) |  | Reason for the outcome |
+| blocking_exception_reason | [string](#string) |  | Reason for blocking exception if applicable |
+| method | [string](#string) |  | HTTP method used |
+| protocol | [string](#string) |  | Protocol used (e.g., HTTP/1.1) |
+| xff_header_value | [string](#string) |  | X-Forwarded-For header value |
+| uri | [string](#string) |  | Request URI |
+| request | [string](#string) |  | Full request |
+| is_truncated | [string](#string) |  | Indicates if the request was truncated |
+| request_status | [string](#string) |  | Status of the request |
+| response_code | [string](#string) |  | HTTP response code |
+| server_addr | [string](#string) |  | Server address |
+| vs_name | [string](#string) |  | Virtual server name |
+| remote_addr | [string](#string) |  | Remote address of the client |
+| destination_port | [string](#string) |  | Destination port |
+| server_port | [string](#string) |  | Server port |
+| violations | [string](#string) |  | List of violations |
+| sub_violations | [string](#string) |  | List of sub-violations |
+| violation_rating | [string](#string) |  | Violation rating |
+| sig_set_names | [string](#string) |  | Signature set names |
+| sig_cves | [string](#string) |  | Signature CVEs |
+| client_class | [string](#string) |  | Client class |
+| client_application | [string](#string) |  | Client application |
+| client_application_version | [string](#string) |  | Client application version |
+| severity | [string](#string) |  | Severity of the violation |
+| threat_campaign_names | [string](#string) |  | Threat campaign names |
+| bot_anomalies | [string](#string) |  | Bot anomalies detected |
+| bot_category | [string](#string) |  | Bot category |
+| enforced_bot_anomalies | [string](#string) |  | Enforced bot anomalies |
+| bot_signature_name | [string](#string) |  | Bot signature name |
+| system_id | [string](#string) |  | System ID |
+| instance_tags | [string](#string) |  | Instance tags |
+| instance_group | [string](#string) |  | Instance group |
+| parent_hostname | [string](#string) |  | Parent hostname |
+| display_name | [string](#string) |  | Display name |
+| violations_data | [ViolationData](#events-v1-ViolationData) | repeated | Detailed violation data |
+
+
+
+
+
+
+<a name="events-v1-SignatureData"></a>
+
+### SignatureData
+SignatureData represents signature data contained within each violation
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| sig_data_id | [string](#string) |  | Signature ID |
+| sig_data_blocking_mask | [string](#string) |  | Blocking mask |
+| sig_data_buffer | [string](#string) |  | Buffer information |
+| sig_data_offset | [string](#string) |  | Offset in the buffer |
+| sig_data_length | [string](#string) |  | Length of the signature match |
+
+
+
+
+
+
+<a name="events-v1-ViolationData"></a>
+
+### ViolationData
+ViolationData represents individual violation details
+
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| violation_data_name | [string](#string) |  | Name of the violation |
+| violation_data_context | [string](#string) |  | Context of the violation |
+| violation_data_context_data | [ContextData](#events-v1-ContextData) |  | Context data associated with the violation |
+| violation_data_signatures | [SignatureData](#events-v1-SignatureData) | repeated | Signature data for the violation |
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 <a name="mpi_v1_common-proto"></a>
 <p align="right"><a href="#top">Top</a></p>
 

internal/collector/securityviolationsprocessor/README.md

@@ -1,5 +1,51 @@
-# SecurityViolations Processor
+# Security Violations Processor
 
-Internal component of the NGINX Agent that processes security violation syslog messages. Parses RFC3164 formatted syslog entries from log records and extracts structured attributes. Successfully parsed messages have their body replaced with the clean message content.
+OpenTelemetry Collector processor that transforms NGINX App Protect security violation syslog messages into structured protobuf events.
 
-Part of the NGINX Agent's log collection pipeline.
+## What It Does
+
+Processes NGINX App Protect WAF syslog messages and transforms them into `SecurityViolationEvent` protobuf messages:
+
+1. Parses RFC3164 syslog messages (best-effort mode)
+2. Extracts CSV formatted data from NAP `secops_dashboard` log profile
+3. Parses XML violation details with context extraction (parameter, header, cookie, uri, request)
+4. Extracts attack signature details
+5. Outputs structured protobuf events for downstream consumption
+
+## Implementation
+
+| File | Purpose |
+|------|---------|
+| [`processor.go`](processor.go) | Main processor implementation, RFC3164 parsing, orchestration |
+| [`csv_parser.go`](csv_parser.go) | CSV parsing and field mapping |
+| [`violations_parser.go`](violations_parser.go) | XML parsing, context extraction, signature parsing |
+| [`xml_structs.go`](xml_structs.go) | XML structure definitions (BADMSG, violation contexts) |
+| [`helpers.go`](helpers.go) | Utility functions |
+
+See individual files for implementation details. Protobuf schema defined in [`api/grpc/events/v1/security_violation.proto`](../../../api/grpc/events/v1/security_violation.proto).
+
+## Requirements
+
+- **Input**: NAP syslog messages with `secops_dashboard` log profile (33 CSV fields)
+- **Output**: `SecurityViolationEvent` protobuf messages
+
+## Testing
+
+```bash
+# Run all tests
+go test ./internal/collector/securityviolationsprocessor -v
+
+# Check coverage
+go test ./internal/collector/securityviolationsprocessor -coverprofile=coverage.out
+go tool cover -html=coverage.out
+```
+
+Test coverage: CSV parsing, XML parsing (5 violation contexts), encoding edge cases, error handling.
+
+## Error Handling
+
+Implements graceful degradation:
+- Malformed XML: Logs warning, continues processing
+- Base64 decode errors: Falls back to raw data
+- Missing fields: Uses empty strings
+- Context inference: Derives from violation names when not explicit