Water quality deterioration often occurs in secondary water supply systems (SWSSs), and increased heavy metal concentrations can be a serious problem. In this survey, twelve residential neighborhoods were selected to investigate the influence of SWSSs on the seasonal changes in heavy metal concentrations from input water to tank and tap water. The concentrations of nine evaluated heavy metals in all groups of water samples were found to be far below the specified standard levels in China. The concentrations of Fe, Mn, and Zn increased significantly from the input water samples to the tank and tap water samples in spring and summer (p < 0.05), especially for the water samples that had been stagnant for a long time. Negative correlations were found between most of the heavy metals and residual chlorine (Fe, Cu, Zn, and As, r = -0.186 to -0.519, p < 0.05). In particular, a high negative correlation was observed between Fe and residual chlorine (r = -0.489 to -0.519, p < 0.01) in spring and summer. Fe and Mn displayed positive correlations with turbidity (r = 0.672 and 0.328, respectively; p < 0.05). In addition, Cr and As were found to be positively associated with some nutrients (NO3−, TN, and SO42−; r = 0.420-0.786, p < 0.01). The material of the storage tanks had little influence on the difference in heavy metal concentrations. Overall, this survey illustrated that SWSSs may pose a chronic threat to water quality and could provide useful information for practitioners.