In this report, we describe a fabrication process of low-cost and highly sensitive SERS substrates by using a simple anodizing setup and a low-energy magnetron sputtering method. The structure of the SERS substrates consists of silver nanoparticles deposited on a layer of anodic aluminum oxide (AAO) template. The fabricated SERS substrates are investigated by a scanning electron microscope (SEM), a transmission electron microscope (TEM), and a confocal Raman spectroscope. We have verified from the surface morphology that the fabricated SERS substrates consist of high-density round-shape silver nanoparticles where their size distribution ranges from 10 to 30 nm on the top and the bottom of nanopores. The surface-enhanced Raman scattering activities of these nanostructures are demonstrated using methylene blue (MB) as probing molecules. The detection limit of 108 M can be achieved from this SERS substrate.