Only a reconstructed turret (1954) occupies its ruins today, but Yoshida castle's history dates back to its establishment by Makino Kohaku in 1505 (when it was known as Imahashi-jō). Conquered by Tokugawa Ieyasu in 1565, it was placed under the rule of Sakai Tadatsugu, only to be given to Ikeda Terumasa by Toyotomi Hideyoshi when Ieyasu shifted to Kantō. The Matsudaira clan controlled the castle following the Battle of Sekigahara, but Yoshida-jō went on to be ruled by numerous other daimyō across the Edo period. The three-tiered tower and some remnants of the wall are all that remain in the park surrounding the castle today.