In the XIth century, the Castle in Cabrières Avignon is built by the family of Adhemar (Vicomte de Cavaillon, first hereditary Orange Counts).
Then in the XIVth century, Henri de Chiabeau becomes the owner of the castle. His daughter, Laura, could be Petrarch's Laura.
In 1490, The castle belongs to Giraud, Baron de Caderousse. For use of its land, he brought vaudoises families from Italian Piedmont.
During the massacres of 1545, the village served as a refuge. Under the command of Lord Oppède and despite its surrending, the Vaud population is massacred. The village is razed.
In 1720, a wall was built to protect the Comtat Venaissin of the black plague coming from Marseille and spreading in the whole Provence. It was called the wall of the plague.
1767 Cabrières passes to the lordship of Grammont-Caderousse until 1807.
Nowadays, the castle is own by a family who keeps it private.