Interestingly, many Romans lived at Salisbury for centuries but no mention of Stonehenge has ever been found in Roman records.
Roman are the likeliest candidates to have taken much of the stone that is missing from the site. In Roman times, of course, such rings were far more commonplace although most werent so large. Perhaps, the Romans just didn't see Stonehenge as anything special, although that doesn't seem like the Romans to me.