Recently, a population of compact main sequence (MS) galaxies exhibiting starburst-like properties were identified in the GOODS-ALMA blind survey at 1.1 mm. Several evolution scenarios were proposed to explain their particular physical properties (e.g., compact size, low gas content, short depletion time). In this work, we aim to study the star formation history (SFH) of the GOODS-ALMA galaxies to understand whether or not the so-called starbursts (SBs) in the MS' galaxies exhibit a star formation activity over the last gigayear that is different from that of the MS galaxies and that could explain their specificity. We use the CIGALE SED modelling code to which we add nonparametric SFHs. To compare the recent SFHs of the galaxies quantitatively, we define a parameter, the star formation rate (SFR) gradient, which provides the angle showing the direction that a galaxy has followed in the SFR-versus-stellar-mass plane over a given period. We show that SB in the MS galaxies show positive or weak negative gradients over the last 100, 300, and 1000 Myr, which is at odds with a scenario where these galaxies would be transitioning from the SB region at the end of a strong starburst phase. Normal GOODS-ALMA galaxies and "SB in the MS" galaxies show the same SFR gradient distribution, meaning that they have similar recent SFHs despite their different properties (compactness, low depletion time). The "SBs in the MS" galaxies manage to maintain their star-formation activity, allowing them to stay within the MS. This points toward a diversity of galaxies within a complex MS.