Green Fuzzing: A Saturation-Based Stopping Criterion using Vulnerability Prediction

Stephan Lipp, Daniel Elsner, Severin Kacianka, Alexander Pretschner, Marcel Böhme, Sebastian Banescu

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

Abstract

Fuzzing is a widely used automated testing technique that uses random inputs to provoke program crashes indicating security breaches. A difficult but important question is when to stop a fuzzing campaign. Usually, a campaign is terminated when the number of crashes and/or covered code elements has not increased over a certain period of time. To avoid premature termination when a ramp-up time is needed before vulnerabilities are reached, code coverage is often preferred over crash count to decide when to terminate a campaign. However, a campaign might only increase the coverage on non-security-critical code or repeatedly trigger the same crashes. For these reasons, both code coverage and crash count tend to overestimate the fuzzing effectiveness, unnecessarily increasing the duration and thus the cost of the testing process. The present paper explores the tradeoff between the amount of saved fuzzing time and number of missed bugs when stopping campaigns based on the saturation of covered, potentially vulnerable functions rather than triggered crashes or regular function coverage. In a large-scale empirical evaluation of 30 open-source C programs with a total of 240 security bugs and 1,280 fuzzing campaigns, we first show that binary classification models trained on software with known vulnerabilities (CVEs), using lightweight machine learning features derived from findings of static application security testing tools and proven software metrics, can reliably predict (potentially) vulnerable functions. Second, we show that our proposed stopping criterion terminates 24-hour fuzzing campaigns 6-12 hours earlier than the saturation of crashes and regular function coverage while missing (on average) fewer than 0.5 out of 12.5 contained bugs.

OriginalspracheEnglisch
TitelISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
Redakteure/-innenRene Just, Gordon Fraser
Herausgeber (Verlag)Association for Computing Machinery, Inc
Seiten127-139
Seitenumfang13
ISBN (elektronisch)9798400702211
DOIs
PublikationsstatusVeröffentlicht - 12 Juli 2023
Veranstaltung32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023 - Seattle, USA/Vereinigte Staaten
Dauer: 17 Juli 202321 Juli 2023

Publikationsreihe

NameISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Konferenz

Konferenz32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023
Land/GebietUSA/Vereinigte Staaten
OrtSeattle
Zeitraum17/07/2321/07/23

Fingerprint

Untersuchen Sie die Forschungsthemen von „Green Fuzzing: A Saturation-Based Stopping Criterion using Vulnerability Prediction“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren