This paper presents a study on modeling the propagation process of topology-aware worms. Topology-aware worms are more intelligent and adaptive to network topologies than other worms, thus are more difficult to control. Due to the complexity of the problem, no existing work has solved the problem of modeling the propagation of topology-aware worms. Our major contributions in this paper are firstly, we propose an innovative logic matrix formulation of the propagation process of topology-aware worms; and secondly, we find, from the applications of the formulation in our experiments, the impacts of two different topologies, namely the simple random graph topology and the pseudo power law topology, on a P2P worm’s mean coverage rate in the P2P overlay network. The proposed innovative logic matrix formulation, which is a discrete time deterministic propagation model of topology-aware worms, can translate the propagation process of topology-aware worms into a sequence of logic matrix operations. Its effectiveness and efficiency are demonstrated by its applications in our experiments.