Motivated by recent results reporting the instability of horizonless objects with stable light rings, we revisit the linearized stability of such structures. In particular, we consider an exterior Kerr spacetime truncated at a surface where Dirichlet conditions on a massless scalar are imposed. This spacetime has ergoregions and light rings when the surface is placed sufficiently deep in the gravitational potential. We establish that the spacetime is linearly, mode unstable when it is sufficiently compact, and in a mechanism associated with the ergoregion. In particular, such instability has associated zero modes. At large multipole number the critical surface location for zero modes to exist is precisely the location of the ergosurface along the equator. We show that such modes do not exist when the surface is outside the ergoregion, and that any putative linear instability mechanism acts on timescales tau greater than or similar to 10(5)M, where M is the black hole mass. Our results indicate therefore that at least certain classes of objects are linearly stable in the absence of ergoregions, even if rotation and light rings are present.