Cardinal Scipione Borghese, (nephew of Pope Paul V) and patron of Bernini,, an important art collector, was the collection's instigator and collected the majority of the collection. His collection was poetically described as early as 1613 by Scipione Francucci. In 1607, the Pope gave the Cardinal 107 paintings which had been confiscated from the studio of the painter Cavalier D'Arpino. In the following year, Raphael's Deposition was removed by force from the Baglioni Chapel in the church of San Francesco in Perugia and transported to Rome to be given to the Cardinal Scipione through a papal motu proprio. In 1682, part of Olimpia Aldobrandini's inheritance entered the Borghese collection; it included works from the collections of Cardinal Salviati and Lucretia d'Este.
Scipione developed a large estate and vineyard on the Pincian hill in Rome into a vast garden and complex of palaces, the Villa Borghese, which are public gardens and museums. A May Day parade is held there open to the public.
(most of this info is from Wikipedia)