TY - GEN
T1 - RandCompile
T2 - 39th Annual Computer Security Applications Conference, ACSAC 2023
AU - Franzen, Fabian
AU - Wilhelmer, Andreas Chris
AU - Grossklags, Jens
N1 - Publisher Copyright:
© 2023 Owner/Author.
PY - 2023/12/4
Y1 - 2023/12/4
N2 - Recently proposed tools such as LogicMem, Katana, and AutoProfile enable a fine-grained inspection of the operating system's memory. They provide insights that were previously only available for Linux machines specifically instrumented for cooperation with virtual machine introspection frameworks. An overly controlling cloud operator can now regularly deep-inspect VMs under their control. In this paper, we investigate how the concept of software diversity can be employed to remove structural information from the Linux kernel to harden it against automated analysis by the aforementioned tools. We employ a mixture of small targeted obfuscations to the memory layout and randomization of the ABI between functions in the Linux kernel as they provide predictable artifacts across different compilers, kernel configurations and the presence of Structure Layout Randomization. We provide an implementation of our ideas in RandCompile, which is composed of a small patch set for the 5.15 Linux LTS kernel and a compiler plugin. RandCompile seeks to remove structural information artifacts, which we call forensic gadgets, to eliminate all leverage points for further analysis of the tools mentioned above. Our approach does not require major modifications to the kernel code base and only has a negligible performance impact (less than 5% percent), which is less than other major security or debugging features enabled by default in the Linux kernel.
AB - Recently proposed tools such as LogicMem, Katana, and AutoProfile enable a fine-grained inspection of the operating system's memory. They provide insights that were previously only available for Linux machines specifically instrumented for cooperation with virtual machine introspection frameworks. An overly controlling cloud operator can now regularly deep-inspect VMs under their control. In this paper, we investigate how the concept of software diversity can be employed to remove structural information from the Linux kernel to harden it against automated analysis by the aforementioned tools. We employ a mixture of small targeted obfuscations to the memory layout and randomization of the ABI between functions in the Linux kernel as they provide predictable artifacts across different compilers, kernel configurations and the presence of Structure Layout Randomization. We provide an implementation of our ideas in RandCompile, which is composed of a small patch set for the 5.15 Linux LTS kernel and a compiler plugin. RandCompile seeks to remove structural information artifacts, which we call forensic gadgets, to eliminate all leverage points for further analysis of the tools mentioned above. Our approach does not require major modifications to the kernel code base and only has a negligible performance impact (less than 5% percent), which is less than other major security or debugging features enabled by default in the Linux kernel.
KW - OS obfuscation
KW - automated profile generation
KW - binary analysis
KW - memory forensics
UR - http://www.scopus.com/inward/record.url?scp=85180158676&partnerID=8YFLogxK
U2 - 10.1145/3627106.3627197
DO - 10.1145/3627106.3627197
M3 - Conference contribution
AN - SCOPUS:85180158676
T3 - ACM International Conference Proceeding Series
SP - 677
EP - 690
BT - Proceedings - 39th Annual Computer Security Applications Conference, ACSAC 2023
PB - Association for Computing Machinery
Y2 - 4 December 2023 through 8 December 2023
ER -