Evaluating and Detecting Semantic Deviations in SIEM Systems Using Specification-Guided Mutation Testing
ID:55
View protection:Participant Only
Updated time:2026-10-04 23:27:58 Views:25
In-person
Abstract
A SIEM pipeline can accept a record yet interpret a field differently from its producer, changing downstream detection output. We evaluate this gap by specification-guided mutation testing on a Kafka/Avro security-analytics platform. The method enumerates 2050 cases over five contracts, five correlation rules, and nine scenarios, derives expected outputs from rule specifications, and attributes each rejection to a validation gate. An audit separates intended semantic mutations from ineffective transformations, serialization failures, and rule controls. Among 50 data mutations that change the input, pass the tested codec round-trip, and are predicted to affect correlation output, existing structural gates block none and the proposed semantic gate blocks 25 (50.0%); on a mechanically expanded variant set it blocks 70/208 (33.7%) output-sensitive cases. Archived runtime experiments add complementary evidence: excluding three runs during an environmental failure, 15 runs contain ten missing target signals and two incorrect correlation keys. Operational silence rests only on the recorded monitoring checks and is not established for every run. The results show a validation-coverage gap in the studied system; they do not estimate attack-detection accuracy or a production false-alarm rate.
Keywords
SIEM,data semantic deviation,mutation testing,data schema,, correlation rules,MITRE ATT&CK
Post comments