According to a legend, in 1421, an Islamic trader named Ayuba had a dream while staying here, near a "Mystic Stone", instructing him to build a mosque. Strangely, when he awoke, he found that the foundations were already in place and he proceeded to construct the mosque until it was completed.[1][2] There is a belief that he left instructions that he should be buried close to the mosque and that after three days, the baobab which would shoot up on his grave was to be preserved from generation to generation. The baobab tree next to the mosque today is reputed to mark the site of Ayuba's grave.[1] The townsfolk of Larabanga supposedly depend on the leaves and stem of this baobab tree for healing of ailments.

Unlike mosques situated in urban settings in West Africa, the Larabanga Mosque is comparatively small. Rural mosques, like Larabanga's, were usually conceived by a single marabout and loosely based on styles seen elsewhere such as in the Great Mosque of Djenné.[3] In order to achieve a physical resemblance to the architecture used elsewhere, the Larabanga Mosque had to incorporate large buttresses in order to compensate for the poorer quality of building materials.[3] Larabanga is one of eight ancient and highly revered mosques in Ghana and is also the oldest. It is a place of pilgrimage and is considered the Mecca of West Africa.[1]

In the 1970s, a mixture of sand and cement was applied to the external faces of the mosque with the intention of protecting the mosque from wind and rain damage. However, this treatment resulted in substantial damage to the building as moisture became trapped in the walls built of adobe and started a deterioration process of the structure, with termites infesting the wooden supports under humid conditions.[4] This resulted in part of the mosque collapsing and during the repair work it caused some distortions of the structural elements and the exterior of the mosque.[4]

Owing to the effect of prevailing winds and rains on the walls, the mosque has needed several renovations and restoration work which over the years have altered some of its exterior designs.[5][6] In September 2002, a severe storm destroyed the mihrab and the minaret. As a result, the World Monuments Fund (WMF) placed the mosque on the 2002 World Monuments Watch,[4] also considering the damage sustained after the inappropriate restoration in the 1970s.[4] The Ghana Museums and Monuments Board decided to restore the mosque and sought advice from CRAterre, an architectural firm based in Grenoble, France which has expertise in building earthen structures.[4] The renovation work was supported by a WMF, with a grant fund of US$50,000 from American Express.[7] The local community also provided support.[4] The conservation process involved removal of the earlier cement plaster from the surfaces of the mosque, the wooden structural components were replaced, the minaret and the mihrab were reconstructed, the portal was redone, and the interior and exterior surfaces were plastered in the traditional way.[4][7] The renovation resulted in reassessing the state of conservation of the site, involving a team of local artisans and labourers. It helped to restore the monument with special emphasis on reviving the knowledge of Adobe maintenance.[4]