Lorenzo and Giovanni di Pier Francesco de` Medici bought the Villa in Castello, situated on the hills around Florence in 1480 from the Della Stufa family. They extended and decorated it with important works of art. In origin, "The Birth of Venus" and "Primavera", two masterpieces by Botticelli nowadays exposed at the Uffizi Gallery should decorate the walls of this villa.

In 1527 the Duke of Urbino rushed to the aid of Florence which was menaced by Charles V`s army, and transformed the villa in his headquarters; then the villa was plundered and burn during Florence siege (1529-1530).

As soon as he became Grand Duke of Tuscany in 1538, Cosimo I commissioned Vasari to renovate the villa and Tribolo to design the wonderful garden that can still be admired today. However, the works for the extension of the villa were never finished because meanwhile Cosimo had bought Palazzo Pitti in Florence

It was Ferdinand I de` Medici who completed the villa, which was then largely renovated under the Lorraine. In particular, in 1818 Leopold II of Habsburg-Lorraine commissioned the realisation of an alley linking the villa in Castello to the neighbouring Villa della Petraia.

After a long period of decadence, in 1919 Victor Emmanuel III gave the villa to the State. In the 1970s the villa was accurately restored and at the present is the headquarters of the Accademia della Crusca.