Gravitational-wave astronomy has the potential to explore one of the deepest and most puzzling aspects of Einstein's theory: the existence of black holes. A plethora of ultracompact, horizonless objects have been proposed to arise in models inspired by quantum gravity. These objects may solve Hawking's information-loss paradox and the singularity problem associated with black holes, while mimicking almost all of their classical properties. They are, however, generically unstable on relatively short timescales. Here, we show that this 'ergoregion instability' leads to a strong stochastic background of gravitational waves, at a level detectable by current and future gravitational-wave detectors. The absence of such background in the first observation run of Advanced LIGO already imposes the most stringent limits to date on black-hole alternatives, showing that certain models of 'quantum-dressed' stellar black holes can be at most a small percentage of the total population. The future LISA mission will allow for similar constraints on supermassive black-hole mimickers.