Efficient Dynamic Flow Tracking for Packet Analyzers

Analyzing large amounts of traffic at the packet or flow level is an important part of managing and monitoring cloud network infrastructure. Common scenarios that require low-level packet analysis are troubleshooting problems, accounting traffic, and security applications such as intrusion detection systems or firewalls. Moreover, researchers often analyze traffic for scientific purposes. For such low-level traffic analyses, tracking flows is a feature required for both commercial and scientific purposes. However, there is no good shared library available to implement this functionality in an efficient, configurable, and dynamic way that is suitable for real-time analysis. We implement a high-performant generic flow tracker that can track millions of simultaenous flows based on arbitrarily complex definitions of a flow. We make this implementation available as open source in our traffic analysis tool FlowScope. The highly efficient realtime tracking of flows by arbitrarily complex user-defined flow criteria and filters is enabled by just-in-time (JIT) compilation of flow tracking rules. The code and evaluation scripts are available as free and open source at.