In 1276, King James II of Majorca made Perpignan the capital of the Kingdom of Majorca. He started to build a palace with gardens on the hill of Puig del rey on the south of the town, the architects were Ramon Pau, Pons Descoll, and Bernat Quer with completion in 1309.

In 1415, the Holy Roman Emperor, Sigismund of Luxemburg, organised a European summit in Perpignan, to convince the Avignon Antipope Benedict XIII to resign his office and take to an end the Western Schism through the Council of Constance. On 20 September 1415, the Emperor met with Pope Benedict XIII at the palace with the King Ferdinand I of Aragon and the delegations of the Counts of Foix, Provence, Savoy, Lorraine, the embassy of the Roman church for the Council of Constance, and embassies from the Kings of France, England, Hungary, Castille and Navarre. The pope refused to resign and to recognise the Pope that the Council had chosen, clashing with the emperor who left Perpignan on 5 November.

Part of the northern wing of the palace was destroyed in a siege in 1502. Following the Treaty of the Pyrenees in 1659, France gained Roussillon, and proceed to develop the defensive features of the palace.

The General Council of the Pyrénées-Orientales bought the Palace and the gardens in 1958, to the Ministry of Defence who keeps the Vauban bastions and 19th century buildings and courts as a garrison.[1]