The timing and causes of Common Era (CE) glacier growth in the Arctic Atlantic region remain elusive. There is mounting evidence of advances that predate the Little Ice Age (1250-1850 CE); this challenges the view that 13th century volcanic eruptions triggered change by spurring sea-ice expansion. Recent climate model simulations indicate this response does not require external forcing under contemporaneous (Pre-Industrial) boundary conditions. Here, we try to reconcile these new insights by combining regional proxy evidence of glacier and sea-ice change with a climate model experiment. Collated recently published reconstructions demonstrate that regional climate shifted towards a colder mean state around 650-950 CE, a period marked by low radiative forcing. Unforced model simulations reproduce the time-transgressive evolution of this response, which emerged east of Greenland and progressed towards Svalbard. The inferred pattern is associated with sea-ice feedbacks, triggered by stochastic atmospheric cooling. We argue that this mechanism may explain the timing and pattern of CE glacier growth in the region.