Distributed event aggregation for content-based publish/subscribe systems

Modern data-intensive applications handling massive event streams such as real-time traffic monitoring require support for both rich data filtering and aggregation. While the pub/sub communication paradigm provides an effective solution for the sought semantic diversity of event filtering, the event processing capabilities of existing pub/sub systems are restricted to singular event matching without support for stream aggregation, which so far can be accommodated only at the subscriber edge brokers. In this paper, we propose the first systematic solution for supporting distributed aggregation over a range of time-based aggregation window semantics in a content-based pub/sub system. In order to eschew the need to disseminate a large number of publications to subscribers, we strive to distribute the aggregation computation within the pub/sub overlay network. By enriching the pub/sub language with aggregation semantics, we allow pub/sub brokers to aggregate incoming publications and forward only results to the next broker downstream. We show that our baseline solutions, one which aggregates early (at the publisher edge) and another which aggregates late (at the subscriber edge), are not optimal strategies for minimizing bandwidth consumption. We then propose an adaptive rate-based heuristic solution which determines which brokers should aggregate publications. Using real datasets extracted from our traffic monitoring use case, we show that this adaptive solution leads to improved performance compared to that of our baseline solutions.