Yu Garden was first built in 1559 during the Ming Dynasty by Pan Yunduan as a comfort for his father, the minister Pan En, in his old age. Pan Yunduan began the project after failing one of the imperial exams. He named the garden Yu Yuan, yu(豫) meaning "pleasing and satisfying to one's parents." His motivation to build this garden is to show filial piety to his parents and make them enjoy in their old age.[1] Thus, Yu Garden is also known as "Garden of Peace and Comfort".[1] However, Yunduan's appointment as governor of Sichuan postponed construction for nearly twenty years until 1577.

In its original form, Yu Garden has many manufactured waterways, paths, rockeries, plants and buildings. The grounds were designed to be a complex of different gardens within one centre, Leshou Tang.[2] From Pan's own records, the garden served as hub for many social activities when he lived there, and those events influenced the texture of Shanghai society.[2] In particular, of Pan Yuduan's journal entries from 1586 through 1601, nearly three quarters of them are related to various public performances that were held in the garden, with the Leshou Tang being the main venue for these events[2].The garden was the largest and most prestigious of its era in Shanghai, but eventually its expense helped ruin the Pans.[3]

The garden was later inherited by Zhang Zhaolin, Pan Yunduan's granddaughter's husband, and then passed to different owners. A section was briefly organised by Zhang Shengqu as the "Academy of Purity and Harmony" (清和書院, Qīng-Hé Shūyuàn) and the Ling Yuan (靈苑, Líng Yuàn, lit. "Spirit Park"), today's East Garden, was purchased by a group of local leaders in 1709. A group of merchants renovated the increasingly decrepit grounds in 1760 and in 1780 the West Garden was opened to the general public.[4]

The gardens suffered damage numerous times during the 19th century. During the First Opium War, the British army used the Huxinting Teahouse as a base of operations for several days in 1842. During the Taiping Rebellion, the Small Swords Society ran its headquarters in the Dianchun Hall; by the time Qing troops recovered the garden, the original structures had nearly all been destroyed. They were damaged again by the Japanese in 1942 before being repaired by Liangshun Han (Rockery Han[5]) appointed by the Shanghai government from 1956 to 1961. They were opened to the public in 1961 and declared a national monument in 1982.[4]