

Department of Computer Science, The University of Auckland, New Zealand. A taxonomy of obfuscating transformations. Christian Collberg, Clark Thomborson, and Douglas Low.Springer International Publishing, 155–174. In Applied Cryptography and Network Security. DynOpVm: VM-Based Software Obfuscation with Dynamic Opcode Mapping. Xiaoyang Cheng, Yan Lin, Debin Gao, and Chunfu Jia.In International Conference on Applied Cryptography and Network Security. DynOpVm: VM-based software obfuscation with dynamic opcode mapping.

In Proceedings of the 26th USENIX Conference on Security Symposium(SEC). Syntia: Synthesizing the Semantics of Obfuscated Code.

The results clearly indicate the possibility of automatic deobfuscation of virtualization-based obfuscation in specific scenarios.

In addition to a theoretical discussion of different types of deobfuscation methodologies, we present an in-depth practical evaluation that compares state-of-the-art virtualization-based obfuscators with currently available deobfuscation tools. In this work, we systematize existing knowledge of automatic deobfuscation of virtualization-protected programs in a novel classification scheme and evaluate where we stand in the arms race between malware authors and code analysts in regards to virtualization-based obfuscation. To enable the automatic detection and analysis of protected malware, new deobfuscation techniques against virtualization-based obfuscation are constantly being developed and proposed in the literature. One of the most advanced techniques for binary obfuscation is virtualization-based obfuscation, which converts the functionality of a program into the bytecode of a randomly generated virtual machine which is embedded into the protected program. Malware authors often rely on code obfuscation to hide the malicious functionality of their software, making detection and analysis more difficult.
