We analyze the size evolution of pebbles accreted into the gaseous envelope of a protoplanet growing in a protoplanetary disk, taking into account collisions driven by the relative sedimentation speed as well as the convective gas motion. Using a simple estimate of the convective gas speed based on the pebble accretion luminosity, we find that the speed of the convective gas is higher than the sedimentation speed for all particles smaller than 1 mm. This implies that both pebbles and pebble fragments are strongly affected by the convective gas motion and will be transported by large-scale convection cells both toward and away from the protoplanet's surface. We present a simple scheme for evolving the characteristic size of the pebbles, taking into account the effects of erosion, mass transfer, and fragmentation. Including the downwards motion of convective cells for the transport of pebbles with an initial radius of 1 mm, we find pebble sizes between 100 mu m and 1 mm near the surface of the protoplanet. These sizes are generally amenable to accretion at the base of the convection flow. Small protoplanets far from the star (>30 au) nevertheless erode their pebbles to sizes below 10 mu m; future hydrodynamical simulations will be needed to determine whether such small fragments can detach from the convection flow and become accreted by the protoplanet.