We present an analysis of the dust attenuation of star-forming galaxies at z = 2.5-4.0 through the relationship between the UV spectral slope (beta), stellar mass (M-*), and the infrared excess (IRX = L-IR/L-UV) based on far-infrared continuum observations from the Atacama Large Millimeter/sub-millimeter Array (ALMA). Our study exploits the full ALMA archive over the COSMOS field processed by the A3COSMOS team, which includes an unprecedented sample of similar to 1500 galaxies at z similar to 3 as primary or secondary targets in ALMA band 6 or 7 observations with a median continuum sensitivity of 126 mu Jy beam(-1) (1 sigma). The detection rate is highly mass dependent, decreasing drastically below log (M-*/M-circle dot) = 10.5. The detected galaxies show that the IRX-beta relationship of massive (logM(*)/M-circle dot > 10) main-sequence galaxies at z = 2.5-4.0 is consistent with that of local galaxies, while starbursts are generally offset by similar to 0.5 dex to larger IRX values. At the low-mass end, we derive upper limits on the infrared luminosities through stacking of the ALMA data. The combined IRX-M-* relation at log (M-*/M-circle dot) > 9 exhibits a significantly steeper slope than reported in previous studies at similar redshifts, implying little dust obscuration at logM(*)/M-circle dot <10. However, our results are consistent with earlier measurements at z similar to 5.5, indicating a potential redshift evolution between z similar to 2 and z similar to 6. Deeper observations targeting low-mass galaxies will be required to confirm this finding.