In the real world, the index of refraction of a refractive object (caustic object) varies across the wavelengths. Therefore, in physically based caustic rendering, we need to take into account spectral information. However, this may lead to prohibitive running time. In response, we propose a two-step acceleration scheme for spectral caustic rendering. Our acceleration scheme takes into account information across visible wavelengths of the scene, that is, the index of refraction (IOR) (caustic object), light power (light), and material reflectance (surface). To process visible wavelengths effectively, firstly we cluster the wavelengths which have similar first refraction (air to caustic object) directions. In this way, all the wavelengths in a cluster can be represented by one light ray during rendering. Secondly, by considering the surrounding objects (their material reflectance from and visible surface area of the caustic objects) and light power, we compute the refinement amount of each wavelength cluster. Our accelerated algorithm can produce photorealistic rendering results close to their reference images (which are generated by rendering every 1 nm of visible wavelengths) with a significant acceleration magnitude. Computational experiment results and comparative analyses are reported in the paper.