Changes in Earth’s orbit set the pace of glacial cycles, but the role of spatial variability in the insolation forcing of global ice volume remains unknown. Here, we leverage the intrinsic dynamical information in empirical records to show that ice volume responded to summer energy at high northern latitudes, as predicted by Milankovitch theory. However, the external forcing of ice volume encompasses insolation signals with a wide range of orbital frequency content, and cannot be fully accounted for by a unique time series. Southern mid-latitude insolation forcing coincides with the position of the subtropical front and the westerlies, which have been implicated in Quaternary climate changes. Dominant forcing modes at northern mid-latitudes are anti-phased with the canonical Milankovitch forcing, consistent with ice volume sensitivity to latitudinal insolation gradients.