diff --git a/blog-service/2025-10-01-security.md b/blog-service/2025-10-01-security.md new file mode 100644 index 0000000000..0c06e81e94 --- /dev/null +++ b/blog-service/2025-10-01-security.md @@ -0,0 +1,12 @@ +--- +title: threatlookup Search Operator (Security) +image: https://help.sumologic.com/img/sumo-square.png +keywords: + - security + - threat intel +hide_table_of_contents: true +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; + +We are happy to announce the `threatlookup` search operator, which allows you to search logs for matches in threat intelligence, providing security analytics to help you to detect threats in your environment. [Learn more](/docs/search/search-query-language/search-operators/threatlookup/). \ No newline at end of file diff --git a/cid-redirects.json b/cid-redirects.json index b734d68aa2..3ed8a9bcaf 100644 --- a/cid-redirects.json +++ b/cid-redirects.json @@ -1569,6 +1569,7 @@ "/cid/0100": "/docs/manage/security/installation-tokens", "/cid/0020": "/docs/manage/health-events", "/cid/0020001": "/docs/security/threat-intelligence/upload-formats", + "/cid/20002": "/docs/search/search-query-language/search-operators/threatlookup", "/cid/0020003": "/docs/security/threat-intelligence", "/cid/0523": "/docs/manage/manage-subscription/upgrade-account/upgrade-sumo-logic-flex-account", "/cid/0524": "/docs/manage/manage-subscription/cloud-flex-legacy-accounts", diff --git a/docs/cse/integrations/configuring-threatq-source-in-cse.md b/docs/cse/integrations/configuring-threatq-source-in-cse.md index fcb67c0e6e..69766b002e 100644 --- a/docs/cse/integrations/configuring-threatq-source-in-cse.md +++ b/docs/cse/integrations/configuring-threatq-source-in-cse.md @@ -7,7 +7,7 @@ description: Learn how to set up a ThreatQ source. import useBaseUrl from '@docusaurus/useBaseUrl'; - +
icon

timeslice()

 diff --git a/docs/search/search-query-language/search-operators/threatip.md b/docs/search/search-query-language/search-operators/threatip.md index c2e32b606a..7762578442 100644 --- a/docs/search/search-query-language/search-operators/threatip.md +++ b/docs/search/search-query-language/search-operators/threatip.md @@ -6,9 +6,7 @@ sidebar_label: threatip The `threatip` operator correlates data in the [Sumo Logic threat intelligence sources](/docs/security/threat-intelligence/about-threat-intelligence/#sumo-logic-threat-intelligence-sources) based on IP addresses from your log data. This provides security analytics that helps you to detect threats in your environment, while also protecting against sophisticated and persistent cyber-attacks. - The `threatip` operator uses the same lookup as the [Threat Intel Quick Analysis app](/docs/integrations/security-threat-detection/threat-intel-quick-analysis) but is simplified for only IP threat lookups. diff --git a/docs/search/search-query-language/search-operators/threatlookup.md b/docs/search/search-query-language/search-operators/threatlookup.md new file mode 100644 index 0000000000..dc0f773d7d --- /dev/null +++ b/docs/search/search-query-language/search-operators/threatlookup.md @@ -0,0 +1,204 @@ +--- +id: threatlookup +title: threatlookup Search Operator +sidebar_label: threatlookup +--- + +The `threatlookup` search operator allows you to search logs for matches in [threat intelligence](/docs/security/threat-intelligence/about-threat-intelligence/), providing security analytics to help you to detect threats in your environment. + +:::note +You can also use the [`threatip`](/docs/search/search-query-language/search-operators/threatip/) search operator to search threat intelligence data based on IP addresses. +::: + +## Syntax + +``` +threatlookup [singleIndicator] [source=""] [include=""] [,, …] +``` + +Where: +* `singleIndicator` returns the single best matching indicator. (In the response, `num_match` indicates how many actual matches there are.) If `singleIndicator` is not specified, all matching indicators are returned. + + Specifying `singleIndicator` sorts the list of matching indicators using the following priority order, then returns the indicator at the top of the list: + 1. Active indicators over expired indicators (if you use `include="all"`). + 1. Higher confidence indicators. + 1. More malicious indicators. + 1. Most recently updated indicators. + + If there's still a tie at this point, the system picks the indicator the back-end database returned first. + +* `source` is the source to search for the threat intelligence indicator. If `source` is not specified, all sources are searched. +* `include` includes either all, only active, or only expired threat intelligence indicators. If `include` is not specified, only active matching indicators are returned. +* `` is the [indicator](/docs/security/threat-intelligence/upload-formats/#normalized-json-format) to look up for a [field name](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/schema/full_schema.md). At least one field name is required. `` is used to add more indicators to look up. Allowed in the filtering are parentheses `()`; `OR` and `AND` boolean operators; and comparison operators `=`, `<`, `>`, `=<`, `=>`, `!=`.
You can filter on the following indicator attributes: + * `actors` + * `confidence` + * `id` + * `indicator` + * `killChain` + * `source` + * `threatType` + * `type` + * `validFrom` + * `validUntil` + +### Response fields + +Query responses return the following fields: +* `confidence` +* `fields` +* `imported` +* `indicator` +* `valid_from` +* `valid_until` +* `source` +* `threat_type` +* `type` +* `updated` +* `num_match` (if `singleIndicator` is used) + +## Examples + +### Simple examples + +``` +_index=sec_record* +| threatlookup srcDevice_ip +| where _threatlookup.confidence > 50 +| timeslice 1h +| count by _timeslice +``` +``` +_index=sec_record* +| threatlookup singleIndicator srcDevice_ip +| where _threatlookup.confidence > 50 +| timeslice 1h +| count by _timeslice +``` +``` +_index=sec_record* +| threatlookup source="mysource" srcDevice_ip +| where _threatlookup.confidence > 50 +| timeslice 1h +| count by _timeslice +``` +``` +_index=sec_record* +| threatlookup dstDevice_ip, srcDevice_ip +| where _threatlookup.confidence > 50 +| timeslice 1h +| count by _timeslice +``` +``` +_index=sec_record* +| threatlookup source="mysource" dstDevice_ip, srcDevice_ip +| where _threatlookup.confidence > 50 +| timeslice 1h +| count by _timeslice +``` +``` +_index=sec_record* +| threatlookup source="mysource" include="active" dstDevice_ip, srcDevice_ip +| where _threatlookup.confidence > 50 +| timeslice 1h +| count by _timeslice +``` + +### Complex examples + +```sql title="Client IP threat info" +_sourceCategory=AWS/WAF {{client_ip}} +| parse "\"httpMethod\":\"*\"," as httpMethod,"\"httpVersion\":\"*\"," as httpVersion,"\"uri\":\"*\"," as uri, "{\"clientIp\":\"*\",\"country\":\"*\"" as clientIp,country, "\"action\":\"*\"" as action, "\"matchingNonTerminatingRules\":[*]" as matchingNonTerminatingRules, "\"rateBasedRuleList\":[*]" as rateBasedRuleList, "\"ruleGroupList\":[*]" as ruleGroupList, "\"httpSourceId\":\"*\"" as httpSourceId, "\"httpSourceName\":\"*\"" as httpSourceName, "\"terminatingRuleType\":\"*\"" as terminatingRuleType, "\"terminatingRuleId\":\"*\"" as terminatingRuleId, "\"webaclId\":\"*\"" as webaclId nodrop +| threatlookup singleIndicator clientip +| where (_threatlookup.type="ipv4-addr" or _threatlookup.type="ipv6-addr") and !isNull(_threatlookup.confidence) +``` + +```sql title="All IP threat count" +_sourceCategory=Labs/AWS/DynamoDB account=* namespace=* "\"eventSource\":\"dynamodb.amazonaws.com\"" +| json "eventName", "awsRegion", "requestParameters.tableName", "sourceIPAddress", "userIdentity.userName" as event_name, Region, entity, ip_address, user +| where Region matches "*" and tolowercase(entity) matches "*" +| where ip_address != "0.0.0.0" and ip_address != "127.0.0.1" +| count as ip_count by ip_address +| threatlookup singleIndicator ip_address +| where (_threatlookup.type="ipv4-addr" or _threatlookup.type="ipv6-addr") and !isNull(_threatlookup.confidence) +| if (isEmpty(_threatlookup.actors), "Unassigned", _threatlookup.actors) as Actor +| sum (ip_count) as threat_count +``` + +```sql title="Use threatlookup in a subquery" +_sourceCategory=weblogs +[subquery:_sourceCategory="Labs/SecDemo/guardduty" "EC2 Instance" "communicating on an unusual server port 22" +| json field=_raw "service.action.networkConnectionAction.remoteIpDetails" as remoteIpDetails +| json field=_raw "service.action.networkConnectionAction.connectionDirection" as connectionDirection +| where connectionDirection = "OUTBOUND" +| json field=remoteipdetails "ipAddressV4" as src_ip +| threatlookup singleIndicator threat| if (_threatlookup.confidence >= 85, "high", if (_threatlookup.confidence >= 50, "medium", if (_threatlookup.confidence >= 15, "low", if (_threatlookup.confidence >= 0, "unverified", "Unknown")))) as malicious_confidence +| where malicious_confidence = "high" +| compose src_ip] +``` + + + +## Format timestamp results + +Timestamps for the following response fields return results as an integer because they use Unix time (also known as *epoch time*): +* `_threatlookup.imported` +* `_threatlookup.valid_from` +* `_threatlookup.valid_until` +* `_threatlookup.updated` + +To convert the timestamp results to a readable output, you must format it in the search itself with [`formatDate`](/docs/search/search-query-language/search-operators/formatdate). For example: + +``` +_index=sec_record* +| threatlookup source="mysource" device_ip +| formatDate(_threatlookup.valid_until, "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'") as valid_until +``` + + diff --git a/docs/security/threat-intelligence/find-threats.md b/docs/security/threat-intelligence/find-threats.md index 61127829b9..589b5db466 100644 --- a/docs/security/threat-intelligence/find-threats.md +++ b/docs/security/threat-intelligence/find-threats.md @@ -37,7 +37,6 @@ To find threats using IP addresses, use the `threatip` search operator. This ope For more information, see [threatip Search Operator](/docs/search/search-query-language/search-operators/threatip/). - - \ No newline at end of file diff --git a/sidebars.ts b/sidebars.ts index a46882c235..62b5adbba3 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -1474,7 +1474,7 @@ module.exports = { 'search/search-query-language/search-operators/sort', 'search/search-query-language/search-operators/substring', 'search/search-query-language/search-operators/threatip', - // 'search/search-query-language/search-operators/threatlookup', + 'search/search-query-language/search-operators/threatlookup', 'search/search-query-language/search-operators/timeslice', 'search/search-query-language/search-operators/timeslice-join', 'search/search-query-language/search-operators/tolowercase-touppercase',