After Chiang Kai-shek's retreat to Taiwan in 1949, Chiang felt it was difficult to accommodate foreign ambassadors due to the lack of five-star hotels in Taipei. He wanted to build an extravagant hotel that would cater to foreign guests. His wife Soong Mei-ling (Madame Chiang) suggested building it on the old Taiwan Hotel on Yuanshan Mountain, the site of the ruins of the Taiwan Grand Shrine, a Shinto shrine during the Japanese rule. Chiang decided on a Chinese palace-style architecture to promote Chinese culture to the West through its extravagance. Taipei-based architect Yang Cho-Cheng was responsible for the design of the new hotel.

The hotel was established in May 1952, but it was expanded several times before it became the landmark as it is known today. The swimming pool, tennis court, and the membership lounge were constructed in 1953, and the Golden Dragon Pavilion and Golden Dragon Restaurant opened in 1956. The Jade Phoenix Pavilion and Chi-Lin Pavilion opened in 1958 and 1963, respectively. In 1968 the hotel was rated as one of the world's top ten hotels by the US Fortune magazine. Finally, on the Double Tenth Day in 1973, the main Grand Hotel building was completed and became an instant Taipei icon.

In June 1995 a disastrous fire broke out on the roof of the main building during necessary reconstruction and refurbishment. As neither ladders nor high pressure pumps could reach the fire, the roof and the upper floors were destroyed. Not until 1998 did the hotel recover from the damage and fully reopen to the public. Following the fire, the two dragon heads on the roof were rotated 180 degrees to point inwards. As dragons are traditionally a symbol of rain and water, this was intended to symbolize preparedness against a future fire.