We study the dynamics of a solar-type star orbiting around a black hole binary (BHB) in a nearly coplanar system. We present a novel effect that can prompt a growth and significant oscillations of the eccentricity of the stellar orbit when the system encounters an "apsidal precession resonance," where the apsidal precession rate of the outer stellar orbit matches that of the inner BHB. The eccentricity excitation requires the inner binary to have a nonzero eccentricity and unequal masses, and can be created even in noncoplanar triples. We show that the secular variability of the stellar orbit's apocenter, induced by the changing eccentricity, could be potentially detectable by Gaia. Detection is favorable for BHBs emitting gravitational waves in the frequency band of the Laser Interferometer Space Antenna, hence providing a distinctive, multimessenger probe of the existence of stellar-mass BHBs in the Milky Way.