Ultralight scalar fields around spinning black holes can trigger superradiant instabilities, forming a long-lived bosonic condensate outside the horizon. We use numerical solutions of the perturbed field equations and astrophysical models of massive and stellar-mass black hole populations to compute, for the first time, the stochastic gravitational-wave background from these sources. In optimistic scenarios the background is observable by Advanced LIGO and LISA for field masses m(s) in the range similar to[2 x 10(-13), 10(-12)] and similar to 5 x [10(-19), 10(-16)]eV, respectively, and it can affect the detectability of resolvable sources. Our estimates suggest that an analysis of the stochastic background limits from LIGO O1 might already be used to marginally exclude axions with mass similar to 10(-12.5) eV. Semicoherent searches with Advanced LIGO (LISA) should detect similar to 15(5) to 200(40) resolvable sources for scalar field masses 3 x 10(-13) (10(-17)) eV. LISA measurements of massive BH spins could either rule out bosons in the range similar to[10(-18), 2 x 10(-13)] eV, or measure m(s) with 10% accuracy in the range similar to[10(-17), 10(-13)] eV.