The atrium of the palace with the statue of pope Julius III by Fulvio Signorini. Photo by Paolo Monti, 1965.

In the mid-twelfth century, the aristocratic Marescotti family of Siena, owners of a castle at the site, erected the tower that stands today next to the palace. Their emblem (an eagle with outstretched wings) is visible on the trefoil windows of the facade.[1] Subsequent reconstructions led to the Gothic structure visible now.

In 1506, the Piccolimini-Mandoli family acquired the building and refurbished the palace interiors, cortile, and loggia in a Renaissance style.

In 1770, the owner Marcantonio Saracini undertook a restoration of the building that lasted until 1824, when Galgano, his son, inherited it. Preserving the original Gothic façade, the castle matches the characteristic curvature of this narrow medieval street. The interior was renovated and modernized by Agostino Fantastici, and art collections were added.

In 1877, after Galgano Saracini died, the palace was inherited by Fabio Saracini, who died in a hunting accident, passing the castle on to his nephew Guido. Count Guido Chigi-Saracini, the castle's last owner, restored the building's exterior and interior with the help of architect Arturo Viligiardi (especially the rococo-style concert hall). In 1932, he founded the Chigiana music academy, to which he bequeathed the building after his death in 1965.