Skip to content

AggregateIPFlows

Jump to bottom Edit New page
Tom Barbette edited this page May 7, 2020 · 3 revisions

AggregateIPFlows Element Documentation

NAME

AggregateIPFlows — Click element; sets aggregate annotation based on flow

SYNOPSIS

AggregateIPFlows([KEYWORDS])

Batching: Batching natively supported
Ports: 1 input, 1-2 outputs
Processing: agnostic, but output 1 is push

DESCRIPTION

AggregateIPFlows monitors TCP and UDP flows, setting the aggregate annotation on every passing packet to a flow number, and the paint annotation to a direction indication. Non-TCP/UDP packets and short packets are emitted on output 1, or dropped if there is no output 1.

AggregateIPFlows uses source and destination addresses and source and destination ports to distinguish flows. Reply packets get the same flow number, but a different paint annotation. Old flows die after a configurable timeout, after which new packets with the same addresses and ports get a new flow number. UDP, active TCP, and completed TCP flows have different timeouts.

Flow numbers are assigned sequentially, starting from 1. Different flows get different numbers. Paint annotations are set to 0 or 1, depending on whether packets are on the forward or reverse subflow. (The first packet seen on each flow gets paint color 0; reply packets get paint color 1. ICMP errors get paints 2 and 3.)

AggregateIPFlows can optionally apply aggregate annotations to ICMP errors. See the ICMP keyword argument below.

If FRAGMENTS is true (the default in push context), AggregateIPFlows assigns aggregate annotations to second and subsequent fragments. It does this by holding onto the fragments until it finds the relevant port numbers, which are attached to the first fragment. Fragments for which no port numbers can be found after FRAGMENT_TIMEOUT seconds of packet time are emitted on port 1 or dropped.

Fragment processing may cause AggregateIPFlows to reorder packets. Packets sent between any given pair of addresses are always emitted in the order in which they were received, but packets sent between different address pairs may come out in a different relative order.

Fragment processing also causes AggregateIPFlows to store packets. If you aren't careful, those stored packets might remain in the AggregateIPFlows element when the driver quits, which is probably not what you want. The Example below shows one way to manage potentially lingering fragments.

If FRAGMENTS is false, second and subsequent fragments are emitted on port 1 or dropped.

Keywords are:

  • TRACEINFO — Filename. If provided, output information about each flow to that filename in an XML format. Only available at userlevel.
  • SOURCE — Element. If provided, the results of that element's 'filename' and 'packet_filepos' read handlers will be recorded in the TRACEINFO dump. (It is not an error if the element doesn't have those handlers.) The 'packet_filepos' results may be particularly useful, since a reader can use those results to skip ahead through a trace file. Only available at userlevel.
  • TCP_TIMEOUT — The timeout for active TCP flows, in seconds. Default is 24 hours.
  • TCP_DONE_TIMEOUT — The timeout for completed TCP flows, in seconds. A completed TCP flow has seen FIN flags on both subflows. Default is 30 seconds.
  • UDP_TIMEOUT — The timeout for UDP connections, in seconds. Default is 1 minute.
  • FRAGMENT_TIMEOUT — The timeout for fragments, in seconds, Default is 30 seconds.
  • REAP — The garbage collection interval. Default is 20 minutes of packet time.
  • ICMP — Boolean. If true, then mark ICMP errors relating to a connection with an aggregate annotation corresponding to that connection. ICMP error packets get paint annotations equal to 2 plus the paint color of the encapsulated packet. Default is false.
  • FRAGMENTS — Boolean. If true, then try to assign aggregate annotations to all fragments. May only be set to true if AggregateIPFlows is running in a push context. Default is true in a push context and false in a pull context.
  • SYMETRIC — Boolean. If true, reverse connections are considered the same connection. Else, every different typle would be considered as separate connection (the paint ANNO will be always 0).

AggregateIPFlows is an AggregateNotifier, so AggregateListeners can request notifications when new aggregates are created and old ones are deleted.

ELEMENT HANDLERS

  • clear (write-only) — Clears all flow information. Future packets will get new aggregate annotation values. This may cause packets to be emitted if FRAGMENTS is true.

EXAMPLES

This configuration counts the number of packets in each flow in a trace, using AggregateCounter.

   FromDump(tracefile.dump, STOP true, FORCE_IP true)
       -> af :: AggregateIPFlows
       -> AggregateCounter
       -> Discard;
   DriverManager(wait, write af.clear)

The DriverManager line handles fragments that might be left in the AggregateIPFlows element after FromDump completes. Again, AggregateIPFlows collects fragments as they arrive and emits them later. If the last packets in FromDump are fragments, then AggregateIPFlows will still be hanging onto them when FromDump stops the driver. The DriverManager element waits for FromDump to request a driver stop, then calls the af.clear handler to flush any remaining fragments.

SEE ALSO

AggregateIP, AggregateIPAddrPair, AggregateCounter, DriverManager

Generated by click-elem2man from ../elements/analysis/aggregateipflows.hh:11 on 2020/05/07.

Add a custom footer
Clone this wiki locally