The original castle was a small fort built around 1320 by Clan MacDougall, who were then Lords of Lorn.[1] Around 1388 the Stewarts took over the Lordship of Lorn, and it is believed that they built the castle in its present form around the 1440s. The Stewarts' relative King James IV of Scotland visited the castle, and a drunken bet around 1620 resulted in the castle passing to Clan Campbell.[1] After the castle changed hands between these clans a couple of times, the Campbells finally abandoned it about 1840 when it lost its roof. In 1908 the castle was bought by Charles Stewart of Achara, who carried out basic conservation work. In 1965 Lt. Col. D. R. Stewart Allward acquired the castle and over about ten years fully restored it. Castle Stalker remains in private ownership and is open to the public at selected times during the summer.[2]

For the 2011 census the island on which the castle stands was classified by the National Records of Scotland as an inhabited island that "had no usual residents at the time of either the 2001 or 2011 censuses."[3]

 

View of Loch Laich and Castle Stalker with the mountains of Morvern in the background