TY - GEN
T1 - Code obfuscation against symbolic execution attacks
AU - Banescu, Sebastian
AU - Collberg, Christian
AU - Ganesh, Vijay
AU - Newsham, Zack
AU - Pretschner, Alexander
N1 - Funding Information:
We thank Saumya Debray and Martín Ochoa for their valuable insights and feedback. Coll-berg was supported by National Science Foundation grants 1525820 and 1318955.
PY - 2016/12/5
Y1 - 2016/12/5
N2 - Code obfuscation is widely used by software developers to protect intellectual property, and malware writers to hamper program analysis. However, there seems to be little work on systematic evaluations of effectiveness of obfuscation techniques against automated program analysis. The result is that we have no methodical way of knowing what kinds of automated analyses an obfuscation method can withstand. This paper addresses the problem of characterizing the resilience of code obfuscation transformations against automated symbolic execution attacks, complementing existing works that measure the potency of obfuscation transformations against human-assisted attacks through user studies. We evaluated our approach over 5000 different C programs, which have each been obfuscated using existing implementations of obfuscation transformations. The results show that many existing obfuscation transformations, such as virtualization, stand little chance of withstanding symbolicexecution based deobfuscation. A crucial and perhaps surprising observation we make is that symbolic-execution based deobfuscators can easily deobfuscate transformations that preserve program semantics. On the other hand, we present new obfuscation transformations that change program behavior in subtle yet acceptable ways, and show that they can render symbolic-execution based deobfuscation analysis ineffective in practice.
AB - Code obfuscation is widely used by software developers to protect intellectual property, and malware writers to hamper program analysis. However, there seems to be little work on systematic evaluations of effectiveness of obfuscation techniques against automated program analysis. The result is that we have no methodical way of knowing what kinds of automated analyses an obfuscation method can withstand. This paper addresses the problem of characterizing the resilience of code obfuscation transformations against automated symbolic execution attacks, complementing existing works that measure the potency of obfuscation transformations against human-assisted attacks through user studies. We evaluated our approach over 5000 different C programs, which have each been obfuscated using existing implementations of obfuscation transformations. The results show that many existing obfuscation transformations, such as virtualization, stand little chance of withstanding symbolicexecution based deobfuscation. A crucial and perhaps surprising observation we make is that symbolic-execution based deobfuscators can easily deobfuscate transformations that preserve program semantics. On the other hand, we present new obfuscation transformations that change program behavior in subtle yet acceptable ways, and show that they can render symbolic-execution based deobfuscation analysis ineffective in practice.
UR - http://www.scopus.com/inward/record.url?scp=85007524382&partnerID=8YFLogxK
U2 - 10.1145/2991079.2991114
DO - 10.1145/2991079.2991114
M3 - Conference contribution
AN - SCOPUS:85007524382
T3 - ACM International Conference Proceeding Series
SP - 189
EP - 200
BT - Proceedings - 32nd Annual Computer Security Applications Conference, ACSAC 2016
PB - Association for Computing Machinery
T2 - 32nd Annual Computer Security Applications Conference, ACSAC 2016
Y2 - 5 December 2016 through 9 December 2016
ER -