Theory and observations suggest that single-star evolution is not able to produce black holes with masses in the range 3-5M (circle dot) and above similar to 45M (circle dot), referred to as the lower mass gap and the upper mass gap, respectively. However, it is possible to form black holes in these gaps through mergers of compact objects in, e.g., dense clusters. This implies that if binary mergers are observed in gravitational waves with at least one mass-gap object, then either clusters are effective in assembling binary mergers, or our single-star models have to be revised. Understanding how effective clusters are at populating both mass gaps have therefore major implications for both stellar and gravitational wave astrophysics. In this paper we present a systematic study of how efficient stellar clusters are at populating both mass gaps through in-cluster mergers. For this, we derive a set of closed form relations for describing the evolution of compact object binaries undergoing dynamical interactions and mergers inside their cluster. By considering both static and time-evolving populations, we find in particular that globular clusters are clearly inefficient at populating the lower mass gap in contrast to the upper mass gap. We further describe how these results relate to the characteristic mass, time, and length scales associated with the problem.