The problem of data latency when network traffic is particularly high

[alleniverson33]

When the network traffic is particularly high, the Suricata alarm shows a delay of one hour in the dashboard. Is there any good solution to this？
I feel like Logstash can't forward it anymore

[mmguero]

I don't understand what you're saying: what do you mean the suricata alarm? Just that the data is taking an hour to show up? Or does the data have incorrect time stamps? What are your system specs? Is it with or without a network sensor? Is the other data (zeek?) showing up correctly?

I'm going to be on vacation until December 2nd, but I will follow up here when I return.

[alleniverson33]

I don't understand what you're saying: what do you mean the suricata alarm? Just that the data is taking an hour to show up? Or does the data have incorrect time stamps? What are your system specs? Is it with or without a network sensor? Is the other data (zeek?) showing up correctly?

I'm going to be on vacation until December 2nd, but I will follow up here when I return.

malcolm k8s
When I attack a target, theoretically Suricata can detect the attack record and display it on the dashboard
Testing in a testing environment is normal, as long as an attack occurs, the attack record can be displayed
But in production environments, network traffic may be particularly high, and attack records may not be real-time. It may take an hour to see the logs of the attack now
I'm not sure if it's a problem with Filebeat collection or if opensearch insertion is experiencing a bottleneck

[mmguero]

Converting to a troubleshooting discussion, we can continue the conversation there.