Reachability Analysis of Concurrent Self-modifying Code
Résumé
We introduce a new effective way to analyze multi-threaded programs that contain self modifying code, i.e., code that modifies itself during its execution. This kind of code is widely used in malware to hide the malicious portions of their code. We introduce a new model called Self Modifying Dynamic Pushdown Network (SM-DPN) to model such programs. A SM-DPN is a network of Self-Modifying Pushdown Systems, i.e., Pushdown Systems that can modify their instructions on the fly during execution. We use finite automata to model regular infinite sets of configurations of SM-DPNs, and propose an algorithm to compute a finite automaton corresponding to the pre* reachable configurations of SM-DPNs. This allows to perform the backward reachability analysis of self modifying multi-threaded programs. We implemented our techniques in a tool and obtained encouraging results. In particular, our tool was able to detect a self-modifying multithreaded malware.
Domaines
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |