Stripped-envelope supernovae (Types IIb, Ib, and Ic) that show little or no hydrogen comprise roughly one-third of the observed explosions of massive stars. Their origin and the evolution of their progenitors are not yet fully understood. Very massive single stars stripped by their own winds (greater than or similar to 25-30 M-circle dot at solar metallicity) are considered viable progenitors of these events. However, recent 1D core-collapse simulations show that some massive stars may collapse directly into black holes after a failed explosion, with a weak or no visible transient. In this Letter, we estimate the effect of direct collapse into a black hole on the rates of stripped-envelope supernovae that arise from single stars. For this, we compute single-star MESA models at solar metallicity and map their final state to their core-collapse outcome following prescriptions commonly used in population synthesis. According to our models, no single stars that have lost their entire hydrogen-rich envelope are able to explode, and only a fraction of progenitors left with a thin hydrogen envelope do (IIb progenitor candidates), unless we use a prescription that takes the effect of turbulence into account or invoke increased wind mass-loss rates. This result increases the existing tension between the single-star paradigm to explain most stripped-envelope supernovae and their observed rates and properties. At face value, our results point toward an even higher contribution of binary progenitors to stripped-envelope supernovae. Alternatively, they may suggest inconsistencies in the common practice of mapping different stellar models to core-collapse outcomes and/or higher overall mass loss in massive stars.