It is uncertain when human settlement first began on Aogashima, but the island was known to be inhabited in the early Edo period, and is mentioned in historical records kept by the Tokugawa shogunate in Hachijōjima. During a major volcanic eruption in 1785, a large number of islanders perished, and the remainder were evacuated to Hachijōjima. An 1835 census reported 241 inhabitants (133 men, 108 women), mostly engaged in fishing.

On April 1, 1940, the island came under the administrative jurisdiction of Hachijō Subprefecture. The population is centered on two hamlets; Yasundogō (休戸郷) in the east and Nishigō (西郷) in the west.