Before being settled the island would be occasionally visited by fishermen from neighbouring islands looking for shelter in a storm or just resting before continuing on their journeys.

It would appear from old maps and descriptions that this island was known by European cartographers and mariners as "Pulo Bardia", indicating that it was first settled by Malayo-Polynesian peoples. The old maps show a chain of three islands aligned north–south and lying off the east coast of the Malay Peninsula. The most northerly and smallest of these islands is marked P. Bardia, the name it had until the early 1900s. The best map example is by John Thornton from The English Pilot, the Third Book, dated 1701, but the specific map of the Gulf of Siam is dated around 1677.[1] Also see maps of the East Indies by William Dampier c.1697.[citation needed] By modern standards of accuracy, the islands are poorly placed on early maps. Seventeenth century marine navigation and cartography used the "backstaff" which, in this area, was accurate to one degree of longitude, or around 60 nautical miles.

The Edinburgh Gazetteer, or Geographical Dictionary published in 1827 also mentions the island and provides a geographical position.[2] In his 1852 book titled Narrative of a Residence in Siam. by Frederick Arthur Neale, the author describes the people and wildlife of Bardia. According to the account there were farms and even cows in a village on the bay lying on the west side of the island. The book includes a fanciful illustration of "Bardia" showing huts and palm trees.[3]

Joseph Huddart in 1801 included these directions for navigating the islands, "To the N.W. by N are two islands of about the same height as Poolo Carnom [Ko Samui]; the first, called SANCORY [Ko Pha-ngan], is 7 leagues from Carnom; the other...,named BARDA, or Bardia [Ko Tao], is 7½ leagues from Sancory."[4] (A league is approximately 3 nautical miles or 5.5 km.)

On 18 June 1899, King Chulalongkorn visited Ko Tao and left as evidence his monogram on a huge boulder at Jor Por Ror Bay next to Sairee Beach. This place is still worshiped today.

In 1933, the island started to be used as a political prison. In 1947 Khuang Abhaiwongse, prime minister at that time, pleaded and received a royal pardon for all prisoners on the island. Everybody was taken to the shore of Surat Thani and Ko Tao was abandoned again.

In the 1980s, overseas travellers began to visit Ko Tao and it quickly became a popular destination. In the 1990s the island became known as a diving site.