The paper is devoted to code optimization techniques with respect to various criteria. Code optimization is well studied for compiled languages; however, interpreted languages can also benefit when using optimization approaches. We provide a work in progress of how the code optimization can be effectively performed for the applications developed with the use of interpreted languages. Methods and techniques proposed in the paper rely on the use of formal models, and in particular state models. We propose some code optimization based on two different state models, namely weighted tree automata, and extended finite automata. The problem of extraction of such models is known to be hard, and in both cases we provide some recommendations of how such models can be derived for a code in an interpreted language. All the optimization techniques proposed in the paper are followed by corresponding illustrative examples